Compare commits

...

2537 Commits

Author SHA1 Message Date
AniLeo 7df8258551 rpcs3_version: Bump to 0.0.10 2020-04-30 19:40:32 +01:00
Eladash 2b75df22d9 sys_event_queue: Fix ports disconnection after queue destruction 2020-04-30 18:58:42 +03:00
Eladash 37110098c7 cellAudioOut Improvements 2020-04-30 15:45:20 +03:00
Megamouse 8f0af6a6fe rsx/interpreter: merge shader settings
- merge disable_asynchronous_shader_compiler and interpreter_mode
- removes disable_asynchronous_shader_compiler setting
- Adds the resulting settings as radio buttons to the gui tab
2020-04-30 15:02:59 +03:00
kd-11 2281c4f662 Fix build 2020-04-30 15:02:59 +03:00
kd-11 2ed50ba263 rsx/interpreter: Improve instruction accuracy
- Fix DIV instruction
- Add EXP_TEX modifier
- Implement WPOS register read
- Swap 3D and Cubemap enums to match RSX ids
- Adds two extra instruction classes: flow control and packing control
- Implement remaining FP instructions with exception of the rare projected texture lookups
- Fix typo causing output color index > 0 to not work
- Fix KIL instruction
- Implement conditional vertex program writes
2020-04-30 15:02:59 +03:00
kd-11 fc5b4026e1 vk: Implement optimized pipeline cache 2020-04-30 15:02:59 +03:00
kd-11 bc5c4c9205 rsx/gl: Implement variable path interpreter for optimal performance 2020-04-30 15:02:59 +03:00
kd-11 930bc9179d rsx/interpreter: Improve instructions support
- Must statically write the gl_ClipDistance registers else you get uninitialized trash.
  This problem is more readily apparent on NVIDIA technology but even AMD is not completely immune.
2020-04-30 15:02:59 +03:00
kd-11 b4bf48c33b vk: Integrate shader interpreter 2020-04-30 15:02:59 +03:00
kd-11 0072df7f20 rsx/gl: Add basic interpreter support to OGL
- Adds basic interpreter functionality.
- Flow control and other instructions not yet implemented.
2020-04-30 15:02:59 +03:00
kd-11 65e9e568b5 config: Register shader interpreter modes 2020-04-30 15:02:59 +03:00
Nekotekina 19acf260d8 SPU DMA: Fix PUTLLUC (TSX)
Prevent edge case of missing store.
2020-04-29 15:40:41 +03:00
Eladash f4f0fb88b1 kernel explorer: Add more information about SPU/PPU threads 2020-04-29 15:32:16 +03:00
Eladash dd6825a7bd Fix sys_ppu_thread_start error checking, fix rare bug in sys_ppu_thread_create
* Correct error code to EBUSY.
* lv2_obj::awake was called even when EBSUY should be returned.
* Fix sys_ppu_thread_create for a newly created thread with the same id as ppu_thread::id_base. (can happen if main thread exited before its creation)
2020-04-29 08:58:09 +03:00
Eladash c1dc6838fa Fix sys_ppu_thread_get_priority page faults 2020-04-29 07:57:47 +03:00
Eladash 69f82a7311 Make spu_mfc_cmd fmt properly show stalled commands 2020-04-29 07:47:49 +03:00
Eladash 0db91aa56b Fix address_range start_length() constructor
Fixes an underflow when constructing start_length(0, 0).
2020-04-29 06:47:35 +03:00
Eladash 791ec95313 sys_rwlock: Do not allow SYS_SYNC_PRIORITY_INHERIT 2020-04-29 05:56:47 +03:00
Eladash 954e3f6e6c Fixup for cpu_flag::pause state check after #8114 2020-04-29 05:56:47 +03:00
Nekotekina d66bdf1653 TSC calibration improvements
Bind main thread to a single core for calibration.
Issue RDTSC after clock probing, may improve accuracy.
2020-04-29 00:09:40 +03:00
Nekotekina 76294beae1 Implement thread_ctrl::get_thread_affinity_mask() 2020-04-29 00:09:40 +03:00
Nekotekina 689419b0ca Remove test_stopped() check from ppu_load_acquire_reservation
Fixes warning.
2020-04-29 00:09:40 +03:00
Eladash 833ace1190 rsx: Fix zcull time to not time travel to the future 2020-04-28 21:07:15 +03:00
Megamouse e095eaf16e Qt: update playtime every 10 seconds 2020-04-28 19:44:09 +02:00
Eladash a505d87565 Partial revert of 3be687cd18 2020-04-28 20:20:19 +03:00
Nekotekina 98ab5d5ba2 atomic.hpp: modernize inline assembly for lock bts/btr/btc
Use flag output (requires clang 9+).
2020-04-28 18:05:32 +03:00
Nekotekina 790fd9ce14 SPU DMA: implement cmp_rdata_avx
Use technique similar to mov_rdata_avx with inline assembly.
2020-04-28 17:58:26 +03:00
Eladash 7da8ba5c15 Wait for SPU event to be received in sys_spu_thread_group_exit
Also use atomic check for group->run_state outside the mutex,
explicitly forbid group termination while we are waiting for an event to be received.
2020-04-28 14:58:17 +03:00
Eladash a9c18964b6 Add missing cpu state check sys_spu_thread_receive_event 2020-04-28 14:58:17 +03:00
Eladash fe7933b0d2 Make ESTAT consistent in sys_spu_thread_group_terminate 2020-04-28 14:58:17 +03:00
Eladash 9506676223 SPU debugger: dump Stall Stat and SRR0 2020-04-28 14:27:40 +03:00
Eladash 7d7909149f SPU: Detect reservation spinning loop 2020-04-28 14:27:40 +03:00
Eladash 3be687cd18 PPU: Fix LWARX/LDARX on TSX path 2020-04-28 14:27:40 +03:00
Eladash e423128a32 SPU: New GETLLAR technique 2020-04-28 14:27:40 +03:00
Nekotekina 3ec73b651e SPU DMA: more tuning for mov_rdata_avx
Avoid unaligned stores.
Prefer asm path if __AVX2__ is not set.
Don't emit vzeroupper if __AVX__ is set.
2020-04-27 18:05:52 +03:00
Nekotekina 4f71c570bd SPU DMA: disable memcpy path
Due to update of the alternative path (SSE/AVX)
2020-04-26 22:36:55 +03:00
Nekotekina 8ae2554505 Implement mov_rdata_avx 2020-04-26 22:36:45 +03:00
Maxim Kulyk c5d390c979 [llvm_build, msbuild] Minor refactor 2020-04-26 18:28:19 +03:00
Megamouse 8e95c0e44d evdev: add keys used by wii controller driver
I'll probably rework the current system sometime soon so that I don't have to add keys every now and then
(or I'll just add them all XD)
2020-04-25 22:55:08 +02:00
Nekotekina 58ba6d68bb Don't use std::popcount (workaround)
It seems MSVC uses POPCNT instruction when compiling for SSE2.
2020-04-25 18:01:39 +03:00
Megamouse 3788ef3e27 evdev: fixup for relax controller criteria 2020-04-25 16:37:20 +02:00
Megamouse b923eb058a Crypto: read sfo in memory instead of tmp file 2020-04-25 15:17:17 +02:00
Megamouse 773448a8f6 Crypto/Qt: check target app version for packages 2020-04-25 15:17:17 +02:00
Megamouse af854835b2 Qt: Rename some functions in settings_dialog 2020-04-25 15:17:17 +02:00
jacob1218 72ab5f05f4 change minimum audio buffer duration 2020-04-25 15:27:18 +03:00
scribam 1791bb5059 Qt: Remove "#pragma once" in a cpp file 2020-04-25 14:56:47 +03:00
scribam 3fd3bd7ca1 spu: Add some "if constexpr" 2020-04-25 14:56:47 +03:00
Megamouse 3937733182 evdev: relax controller criteria 2020-04-25 10:50:38 +02:00
Megamouse de58f19866 input: add Rock Revolution Drum Controller product info 2020-04-25 10:17:48 +02:00
Megamouse e4cb9ef7cd cellpad: add pclass_profile flags 2020-04-25 10:17:48 +02:00
Megamouse 4e6d95c5b8 Qt/input/cellpad: enable product choice 2020-04-25 10:17:48 +02:00
Eladash 256c74def2 sys_rsx: Fix error code instead of success on invalid context 2020-04-23 14:01:04 +03:00
Eladash c48ccc6f3c sys_rsx: Fix zcull bind error checking regression 2020-04-23 14:01:04 +03:00
Megamouse 18219afbf7 Qt: move rsx capture to Utilities menu 2020-04-22 21:43:03 +02:00
Megamouse 1805cb44e6 Qt: move GetBootConfirmation to gui_settings 2020-04-22 21:43:03 +02:00
Megamouse b4b8c1e4b2 Qt: Add confirmation dialogs on drag and drop 2020-04-22 21:43:03 +02:00
Megamouse 193837298b Qt: enum class drop_type 2020-04-22 21:43:03 +02:00
Megamouse 18e0b83ac9 Qt: some cleanup 2020-04-22 16:58:20 +02:00
Megamouse 1a374126e1 Qt: move GetSettingName to cfg_adapter 2020-04-22 16:58:20 +02:00
Megamouse 2b6afb6916 Qt: Add confirmation dialogs before closing games 2020-04-22 16:58:20 +02:00
Megamouse ebd92a2f2f Qt: Add Firmware Cache options to main window menu 2020-04-22 16:58:20 +02:00
Eladash 8c747bf0a2 sys_rsx: More error checks for ZCULL area binding
And clamp zcull offset to 256MB, it's unknown if only the error check clamps or it is clamped entirely.
2020-04-21 16:18:32 +01:00
Eladash b94e4247cc rsx: More strict zcull stats enabling 2020-04-21 16:18:32 +01:00
Megamouse a203ff677b settings: remove legacy settings 2020-04-20 20:56:07 +02:00
Megamouse c3af19148f settings: fix clocks scale default 2020-04-20 20:56:07 +02:00
Eladash dbce10d0e3 PPU LLVM: Fix rounding regression of FNMADDS, FNMSUBS (#8066)
* PPU LLVM: Fix rounding regression of FNMADDS, FNMSUBS
2020-04-19 20:55:26 +01:00
rxys 5101bc189e Fix FMA copypasta (#8060) 2020-04-19 19:17:19 +01:00
Eladash 5960de2e20 PPUAnalyzer: Check if TOC from OPD is a valid address 2020-04-19 10:56:42 +01:00
Eladash 1cb3bf6dab Minor fixup for unimplemented syscall args dump 2020-04-19 10:56:42 +01:00
Eladash 0bf73ba0bc PPU debugger: report functions on registers display 2020-04-19 10:56:42 +01:00
Eladash 368bd7cf02 PPU debugger: read 32-bit pointer instead of 64-bit
PPU ABI supports only 32-bit pointers in userland, also fix it to use super ptr.
2020-04-19 10:56:42 +01:00
Eladash 83c7f6f149 debugger: Rephrase "Current function" to "In function"
Takes less space which makes actual function name display a bit nicer.
Also the meaning is clearer.
2020-04-19 10:56:42 +01:00
Megamouse 2094e52d7d Crypto: add another file type for PSVita 2020-04-18 19:11:53 +02:00
Megamouse 5657363642 Crypto: move PSVita keys to key vault 2020-04-18 19:11:53 +02:00
Megamouse d35a29bbe4 Crypto: PSVita metadata and missing entry type 2020-04-18 19:11:53 +02:00
Megamouse 22d01e4d05 Crypto: interpret metadata versions 2020-04-18 19:11:53 +02:00
Megamouse 1762324702 Crypto: fix metadata variable names
header_size was just wrong, on PS3 games it just happens to match the meta offset, which is half of data_offset
PS3: meta_offset + meta_size = data_offset
PSP: meta_offset + meta_size = metadata_header_hmac_offset
PSP: metadata_header_hmac_offset + ext_data_size = data_offset
2020-04-18 19:11:53 +02:00
Megamouse e4b6de409a Crypto: fix magical type 2020-04-18 19:11:53 +02:00
Megamouse 3a6bda4d93 Crypto: read and log PSP and PSVita headers 2020-04-18 19:11:53 +02:00
Megamouse 3b9dc29781 Crypto: add log verbosity to pkg installations
also enables installation with one more filetype
2020-04-18 19:11:53 +02:00
Eladash 6210507a37 sys_rsx: Fix tiles on MAIN memory error checking 2020-04-18 10:20:03 +03:00
Megamouse 0df6c41556 Qt: move code from emu_settings to config_adapter 2020-04-17 15:46:46 +02:00
Megamouse 7ba5f1f503 Qt: adjust max llvm thread tooltip 2020-04-17 13:30:10 +02:00
Megamouse 171367fe88 Qt: fix localization in change_microphone_type
Don't rely on localized text at all. Use the setting's index and formatted string instead
2020-04-17 13:30:10 +02:00
Megamouse ec4e8eda04 Qt: implement GetIsDynamicConfig in emu_settings
- unused at this point
2020-04-17 13:30:10 +02:00
Megamouse e361bac945 Qt: minor cleanup 2020-04-17 13:30:10 +02:00
Eladash 06d4505992 sceNp: Override k_licensee for PE game category
Co-Authored-By: AniLeo <ani-leo@outlook.com>
2020-04-17 11:41:50 +01:00
Eladash a3f2dfa232 sys_isolated_spu 2020-04-17 11:41:50 +01:00
Eladash 921b1aadfb lv2: Log all arguments of unimplemented syscalls 2020-04-17 11:41:50 +01:00
rexys 8f3b04cbd6 rsx: Fix is_fifo_idle with hle gcm 2020-04-16 12:59:19 +03:00
Nekotekina 0f6a0d2740 Expand vm::g_addr_lock to 64 bit to support ranges
Optimization.
2020-04-16 02:25:43 +03:00
Nekotekina c7fe8567b8 Experimental squashing of reservation memory area.
Enables trivial synchronization between shared mem.
Reduces memory usage, but potentially degrades performance.
Rename an overload of vm::passive_lock to vm::range_lock.
2020-04-16 02:25:43 +03:00
Eladash 8cb1f4fe26 liblv2 HLE: Fix spu_elf_info loader for SCE SPU images (#8045) 2020-04-15 21:19:46 +01:00
Megamouse cf229a8e9f some more dynamic settings 2020-04-15 18:25:25 +02:00
Nekotekina 074b9f94db Fix regression in SPU ASMJIT
Incorrect arithmetics.
2020-04-15 12:19:15 +03:00
scribam 20f53e65eb cmake: Add support for target_precompiled_headers if available 2020-04-14 23:00:51 +03:00
Nekotekina b1b67a13c6 Revert "Replace rotate utils with std::rotl" (partial)
This reverts commit 4d8bfe328b.
2020-04-14 19:45:53 +03:00
Eladash bc3b70c338 Fix ppu_rotate_mask 2020-04-14 19:10:30 +03:00
Eladash 63be05d5d3 minor ppu fixup
does not affect anything except consistency.
2020-04-14 17:09:58 +03:00
Eladash ec1e82bc9d SPU debugger: Implement blocking functions dumping 2020-04-14 17:09:58 +03:00
scribam 2e397e38a4 Typos 2020-04-14 17:06:58 +03:00
scribam 7e0bc26241 Add missing break in cheat_manager.cpp 2020-04-14 17:06:58 +03:00
scribam f37adc4188 Add fallthrough attribute 2020-04-14 17:06:58 +03:00
Nekotekina 4d8bfe328b Replace rotate utils with std::rotl
More include cleanup.
2020-04-14 16:05:58 +03:00
Nekotekina f72af2973d Replace utils::popcnt32 with std::popcount
Cleanup includes.
2020-04-14 16:05:58 +03:00
Nekotekina 032e7c0491 Replace utils::cntlz{32,64} with std::countl_zero 2020-04-14 16:05:58 +03:00
Nekotekina d0c199d455 Replace utils::cnttz{32,64} with std::countr_{zero,one}
Make #include <bit> mandatory.
2020-04-14 16:05:58 +03:00
JohnHolmesII adfc9d93c3 CI: Auto update LLVM and GLSLANG when those releases change (#8021) 2020-04-14 08:15:05 +01:00
Eladash 2dcc3255b2 Fix sys_net_bnet_sendto (#8026) 2020-04-13 18:49:12 +01:00
JohnHolmesII 167159698d Fix overloaded virtual warning 2020-04-13 14:37:11 +03:00
JohnHolmesII 4838f72e50 Prevent unecessary copy in loop 2020-04-13 14:37:11 +03:00
JohnHolmesII a8a83c9724 cellRtc: Extend before shift, per decompiled output 2020-04-13 14:37:11 +03:00
JohnHolmesII f0b1c8302a Fix order of operations warning 2020-04-13 14:37:11 +03:00
Eladash c8b8cafeec PPU: Merge reservations store functions into one 2020-04-13 14:34:37 +03:00
RipleyTom 791c0da236 Add Disney Portal to passthroughs (#8022) 2020-04-13 11:49:42 +01:00
Eladash b45d836b89 sys_net: Fix sys_net_bnet_bind page faults
+ EINVAL checks
2020-04-13 04:34:10 +01:00
Eladash 492a80f6c5 CPUThread.cpp: Minor indetation fixup 2020-04-13 04:34:10 +01:00
Eladash 8f32d44635 sys_net: Fix sys_net_bnet_sendto 2020-04-13 04:34:10 +01:00
Eladash 9fb30f130a sys_net: Fix sys_net_bnet_getsockopt
+ EINVAL checks
2020-04-13 04:34:10 +01:00
Eladash d91d420981 sys_net: EINVAL check in sys_net_bnet_listen 2020-04-13 04:34:10 +01:00
Eladash 442035c251 sys_net: EINVAL checks in sys_net_bnet_accept 2020-04-13 04:34:10 +01:00
Eladash 063902728b sys_net: EINVAL checks in sys_net_bnet_recvfrom 2020-04-13 04:34:10 +01:00
Eladash 60a63fa4b6 sys_net: Fix sys_net_bnet_getsockname page faults
+ EINVAL checks
2020-04-13 04:34:10 +01:00
Eladash 7399a3f1e9 sys_net: Fix sys_net_bnet_getpeername page faults
+ EINVAL checks
2020-04-13 04:34:10 +01:00
Eladash c4f6968aae sys_net: Fix sys_net_bnet_connect page faults
+ EINVAL checks
2020-04-13 04:34:10 +01:00
Eladash 00957ca4bf sys_net: Fix sys_net_bnet_select page faults 2020-04-13 04:34:10 +01:00
Eladash 179a9b3bf0 sys_net: Fix sys_net_bnet_poll page faults 2020-04-13 04:34:10 +01:00
Eladash 926e0467cf Another ::as_rvalue fixup 2020-04-13 04:34:10 +01:00
sampletext32 c69691f19b Fix various explicitness, laziness, hard codes 2020-04-12 17:29:42 +03:00
Nekotekina 5524cd1e75 Improve TAR loader
Don't overwrite unchanged files.
Print error if failed to overwrite.
This commit affects PS3 firmware installation.
Trying to workaround a bug where some files cannot be overwritten.
2020-04-12 16:56:21 +03:00
Nekotekina 17f3a114be Fat atomics: implement exchange() and compare_exchange()
Also includes compare_and_swap() and compare_and_swap_test().
Also includes fixes for load(), store(), and atomic_op().
2020-04-12 16:56:21 +03:00
Eladash cb14805d78 rsx fp/vp analyzers: Fix strict type aliasing and improve codegen 2020-04-12 16:48:43 +03:00
Eladash ae1ff1e96d ppu exec loader: Log TLS image information 2020-04-12 10:30:38 +01:00
Eladash c3a4e57efe Reduce log level of page fault notifications
Log current hle function.
2020-04-12 10:30:38 +01:00
Eladash bb950cbb3b vm: Fix possible IDM deadlock with Page Fault Notifications (partial) 2020-04-12 10:30:38 +01:00
Whatcookie 6b0f7a8f55 PPU LLVM: Optimize altivec FMA with 0 addend (#8013)
- When VMADDFP and VNMSUBFP are used with a constant addend of 0, they can be simplified into a single floating multiply
2020-04-12 09:52:21 +01:00
Eladash 8e61c65c0d Fixup ::as_rvalue 2020-04-11 22:55:55 +03:00
Eladash 141d62fbf9 Implement ::as_rvalue 2020-04-11 21:58:36 +03:00
Eladash e407018bb5 rsx: Write ref+get atomically
May contribute to better FIFO synchronization in some cases.
2020-04-11 21:21:15 +03:00
Eladash ff74c241c7 rsx: Fix get_optimal_blit_target_properties for local memory 2020-04-11 21:21:15 +03:00
Eladash d69bec8f59 rsx: Fix vblank thread stop regression 2020-04-11 21:21:15 +03:00
Eladash 504ba8d824 rsx: Fix grammer issue (binded -> bound) 2020-04-11 21:21:15 +03:00
Eladash 8228fa1ece sys_rsx: Warn if RSX is not idle during crucial points 2020-04-11 21:21:15 +03:00
Eladash 1f9f455801 sys_rsx: Implement error checks for Zcull/Tiles binding
* Check zcull/tiles offset if bigger than max MAIN/LOCAL size.
* Check memory mapping of offset if location is MAIN.
* Check pitch/size for 0 as coming from hw tests.
* In addition: fix 'bound' check of tiles, seem to rely on the bits location is in.
* Add locks for zcull/tiles/displaybuffer binding.
2020-04-11 21:21:15 +03:00
Eladash 5ba26e247b sys_rsx: Implement LLE cellGcmSysGetLastVBlankTime 2020-04-11 21:21:15 +03:00
Eladash 93b8f3b5db idm: Minor update to use std::static_pointer_cast 2020-04-11 10:58:24 +03:00
RainbowCookie32 11b980c9ac Show state of Accurate LLVM DFMA option in GUI for CPUs that support FMA 2020-04-11 10:48:51 +03:00
illusion df20410cf1 gui: don't allow cpu with fma support disable accurate path 2020-04-09 19:22:04 +03:00
Eladash d451a0b7b7 SPU LLVM: Improve FNMS
Should be more accurate with postive/negative zero inputs according to docs while being more optimized.
TODO: Check SPU precise interptreter.
2020-04-09 17:27:14 +03:00
Eladash 158b24ec25 SPU LLVM: Add accurate double-precision FMA support 2020-04-09 17:27:14 +03:00
Nekotekina 1b68f90e42 Tweak TSC calibration
Round to 3 digits after dot (count in MHz).
2020-04-09 16:23:33 +03:00
clienthax 765b14a8ba Implement cellRtc HLE (#7933) 2020-04-09 15:54:41 +03:00
Eladash 36fd1d0f0d rsx: Optimize transform constants load methods (#7992) 2020-04-09 15:53:43 +03:00
JohnHolmesII e8f9fd5430 CI: Unify spacing for build scripts 2020-04-09 13:02:07 +03:00
JohnHolmesII c4a21438ad CI: Maintenance
- Rename .travis dir to .ci, since it isn't just for Travis
- Convert Linux build scripts to posix sh
- Clean up some scripts per shellcheck
2020-04-09 13:02:07 +03:00
JohnHolmesII c2ae8be3eb CI: Fix facepalm bug in Azure build 2020-04-09 13:02:07 +03:00
JohnHolmesII 6fe671a5e8 CI: Prevent Azure from publishing empty files 2020-04-09 13:02:07 +03:00
Eladash cce946f10e spu: typo fix 2020-04-08 19:23:13 +03:00
Eladash c948c9305c sys_ppu_thread_create: read function descriptor immediately and save it 2020-04-08 19:23:13 +03:00
Megamouse 8c838698af Qt: fix time played 2020-04-08 19:20:41 +03:00
Eladash c0f4fb6e35 SPU debugger: dump reservation address 2020-04-08 14:35:44 +03:00
Eladash cc8f024c6c Fixup ppu/spu_thread::dump_all() 2020-04-08 14:35:44 +03:00
Megamouse 2e18df7223 Qt: fix renderer translation
move render creator to own class
2020-04-08 11:43:48 +02:00
Eladash 995fa63f4c sys_rsx: Fixup after #7978 2020-04-08 11:33:26 +03:00
RipleyTom dce81d4a66 fix AddLddPad when more than one ldd pad 2020-04-08 07:39:54 +03:00
Eladash f7536bbce0 sys_rsx: Fix gcm events spam
In realhw the events are only sent if they are masked in driver_info->handlers as well.
2020-04-07 20:43:28 +03:00
Eladash 3f48450408 sys_rsx: Minor atomicity fixes 2020-04-07 20:43:28 +03:00
Víctor "IlDucci a38d2461c9 Linguistic changes (#7917) 2020-04-07 17:10:04 +02:00
Nekotekina 6c8d844ec5 PPU LLVM: fix crash on damaged cache files 2020-04-07 16:51:35 +03:00
Nekotekina 91d80aa7b9 Implement jit_compiler::check
Instead of checking file existence (because file may be damaged).
2020-04-07 16:09:47 +03:00
Nekotekina 7939160178 Shorten SPU LLVM Worker name to SPUW.#
For debugging.
2020-04-07 15:55:30 +03:00
Nekotekina 3eabec0030 SPU: Fix SPU Precise interpreter 2020-04-07 15:42:27 +03:00
Eladash 8f3ad8b81a Fix cellGameDataCheckCreate2 error broken check (#7976) 2020-04-07 14:03:03 +03:00
Eladash 5834a466cd Fix utils::get_tsc_freq() (#7974)
Use magic static for once-initialization
2020-04-07 11:02:12 +03:00
Megamouse 4ff69dc0cd Qt: fix mic_none and move microphone creator code 2020-04-07 08:10:56 +02:00
Megamouse 4aae9a17c1 Qt: make trophy type translateable 2020-04-07 00:26:30 +02:00
Megamouse 5e6928a182 Qt: add disambiguations for settings translations
This prevents that the Qt linguist omits duplicate strings, which are actually supposed to be individually translateable.
2020-04-07 00:26:30 +02:00
Megamouse cc6a03cbd7 Qt: mic_none and enter_button_assign translations 2020-04-07 00:26:30 +02:00
Nekotekina 3c3ccdbf1e Round TSC calibration result towards speculated CPU base frequency 2020-04-07 00:10:08 +03:00
Nekotekina 15f01a1bf6 Implement TSC calibration
Try to get rough TSC frequency by sampling it.
2020-04-07 00:10:08 +03:00
Dzmitry Malyshau b6e52ad975 Fix CMake path to IOKit 2020-04-06 23:23:11 +03:00
Megamouse 2bd4485082 Qt: make cheat_type combobox translateable 2020-04-06 20:59:58 +02:00
Megamouse 96086d57fa Qt: implement EnhanceRadioButton 2020-04-06 20:59:58 +02:00
Megamouse 078c31c1da Qt: fix lupdate warnings (used for translation) 2020-04-06 20:59:58 +02:00
Megamouse 7a409af0b0 Qt: pad handlers translateable 2020-04-06 20:59:58 +02:00
Megamouse e6a6d7e9bc Qt: fix some translation nitpicks 2020-04-06 20:59:58 +02:00
Megamouse 133e897c8b Qt: make comboboxes in settings dialog translateable 2020-04-06 20:59:58 +02:00
Megamouse b1fdbc7fcc Move some format functions 2020-04-06 20:59:58 +02:00
Megamouse 89f16548f3 Qt: const, const everywhere 2020-04-06 20:59:58 +02:00
Eladash e7d5d17fd8 rsx: Adjust FIFO recovery to be a bit more merciful 2020-04-05 17:40:23 +03:00
Eladash bbbd06dcee Make vm::_test_map aware of overflow
+ remove vm::find_map 1024mb limit.
2020-04-05 17:40:23 +03:00
Silent fd09dde911 cellSearch: Change search state before invoking callbacks 2020-04-05 16:35:56 +03:00
kd-11 0b6e2b26fa rsx: Fix DST instruction
- It's the old-school distance vector, not the more modern distance() function
- There is seemingly no glsl function that maps to it directly.
2020-04-05 16:35:20 +03:00
kd-11 b301fecfd8 gl: Fix async shader compiler
- Removes glFinish hack.
- Adds proper server-side synchronization.
- Adds primary context detection to allow worker threads to be identified.
2020-04-05 16:35:20 +03:00
Eladash dc5cdb3bb4 sys_ppu_thread: reduce global memory stats after thread creation 2020-04-05 15:23:09 +03:00
Eladash 72d1efa383 rsx: Batch transform contants load methods 2020-04-05 15:21:56 +03:00
RipleyTom f36686b1a7 Always launch rpcs3.exe on restart 2020-04-05 14:27:13 +03:00
RipleyTom f33373ca1b Add Lego Dimensions Portal to usb passthroughs 2020-04-05 14:23:54 +03:00
Nekotekina ae140a1ac9 SPU LLVM: improve stack mirror format in Mega mode
Store first instruction for additional validation.
Should fix some problems.
I didn't touch ASMJIT.
2020-04-04 21:40:42 +03:00
Nekotekina 8053d2602a SPU LLVM: implement bisect helper (debugging tool)
Added two new settings: SPU LLVM Lower/Upper Bound
By manipulating values, conditionally avoid compiling programs.
It uses hash of the programs (64-bit start hash of SHA-1).
Programs which aren't compiled run with interpreter.
2020-04-04 21:38:40 +03:00
Nekotekina 18dbd788e6 SPU DisAsm: fix disasm for BINZ and similar instruction 2020-04-04 21:38:40 +03:00
Nekotekina a53d0d50b3 SPU LLVM: add alternative ROTQBY implementation
Used if SSSE3 is not available (exec_rotqby).
2020-04-04 21:38:40 +03:00
Nekotekina 7f9d41ac47 Implement cfg::uint for unsigned integers 2020-04-04 21:38:40 +03:00
Nekotekina f05e24e8e8 SPU LLVM: make LS loads/stores volatile
Fixes PS1 classics and possibly something else.
2020-04-04 21:38:40 +03:00
Megamouse cd64990558 Qt: fix nullptr 2020-04-04 21:38:26 +03:00
Eladash 0b24b09a06 sys_fs: Lock dev_hdd1 mount point at cellSysCacheClear 2020-04-04 19:36:16 +01:00
Eladash a178374052 sys_fs: Limit NPDRM FDs to 16
Co-Authored-By: Silent <cookieplmonster@users.noreply.github.com>
2020-04-04 19:36:16 +01:00
Whatcookie dd8a3eaac5 Util: Add FMA and INVARIANT_TSC detection (#7937) 2020-04-04 19:12:06 +01:00
Eladash 63080c22a3 Fix ppu_thread::dump_callstack() 2020-04-04 00:06:51 +03:00
Eladash 55baaea8e7 liblv2 HLE: Fix sys_lwcond_signal mode 3 2020-04-03 19:42:25 +03:00
Eladash 13820d6802 SPU debugger: Show channels data 2020-04-03 18:37:21 +03:00
Eladash 0beea91d5e Minor debugger fixups 2020-04-03 18:37:21 +03:00
Eladash 3f559cd86e Fix sys_cond_destroy (#7931)
Dereference cond count in sys_cond_destroy
2020-04-03 12:45:59 +03:00
Nekotekina 39796141fc Allow AppImage to spawn its own rpcs3 process for fatal error dialog (Linux) 2020-04-03 12:32:05 +03:00
illusion 7c972c8860 Add accurate PPU FMA to advanced tab (#7915) 2020-04-03 03:20:33 +01:00
Nick Renieris 2fb600e458 Qt/Debugger: Don't move entire list if it's not needed
With 4 buffer spaces at the bottom.
2020-04-03 01:36:35 +01:00
Nick Renieris 6cbb12e5cd PPUThread: String & hex previews for register pointers in register dump 2020-04-03 01:36:35 +01:00
Nick Renieris 2eea18469d Qt/Debugger: Call Stack panel 2020-04-03 01:36:35 +01:00
Nick Renieris 1113221340 Qt/MemoryViewer: Make it vertically resizable 2020-04-03 01:36:35 +01:00
Nick Renieris 9024ba69b4 Qt/Debugger: Split register misc state info to separate panels 2020-04-03 01:36:35 +01:00
Nick Renieris 1231274e0f CPUThread: Split dump() info to separate methods 2020-04-03 01:36:35 +01:00
Eladash 72c0aed4c1 rsx: Reset vertex program/constants at each boot 2020-04-02 20:42:12 +03:00
Eladash c2c5005278 rsx: Fix and improve fp program data invalidation 2020-04-02 20:42:12 +03:00
Eladash 2ed370093e rsx: Get rid of invalid_command_interrupt_raised 2020-04-02 20:42:12 +03:00
Eladash d97e9f7b4a rsx: Batch vertex program load methods 2020-04-02 20:42:12 +03:00
EmulationChannel 85c4321c24 Update FW 4.86 Latest Version
Updates the latest FW version according to: https://www.playstation.com/en-us/support/system-updates/ps3/

    What's New in Version 4.86
* This system software update improves system performance.
2020-03-31 22:37:30 +03:00
Nekotekina ba7f4af02b CFG: minor cleanup 2020-03-31 21:50:23 +03:00
kd-11 69d90f6fec vk: Remove NVIDIA workaround for broken partial occlusion queries
- This bug has been fixed in the latest drivers.
2020-03-31 20:53:12 +03:00
kd-11 8c847d3a4b vk: Remove RADV workaround regarding renderpass barriers
- The situation was clarified in the official vulkan spec to allow this
  behavior.
  Barriers are now only inserted by the driver when layout
transitions are requested.
2020-03-31 20:53:12 +03:00
kd-11 b327e329d6 vk: Avoid query log spam if no program is loaded 2020-03-31 20:53:12 +03:00
Eladash 92f821aeb1 PPU LLVM: Add FMA accuracy setting (#7874)
* PPU LLVM : Match PS3 for the instructions fmadd, fmadds, fmsub, fmsubs, fnmadd, fnmadds, fnmsub, fnmsubs

Co-authored-by: doesthisusername <yfirestorm@gmail.com>
2020-03-31 20:01:10 +03:00
Megamouse fc3a134e7d Emu: make "Silence All Logs" dynamic 2020-03-31 18:06:37 +02:00
Megamouse d6b8213c9f set key_vault log spam to trace 2020-03-31 18:06:37 +02:00
Megamouse f079eb4026 add cacert.pem to gitignore 2020-03-31 18:06:37 +02:00
Megamouse 28bea14d72 add missing files to visual studio project 2020-03-31 18:06:37 +02:00
Megamouse a76a4d8136 change sig_log to SIG 2020-03-31 18:06:37 +02:00
Eladash fdd7f0645d Some typos (#7908)
* sys_lwcond: replace writer lock with reader lock

* sys_rsx: Typo fix

* sys_net: Fixup for buffer reading
2020-03-31 16:44:50 +03:00
Eladash 29be815302 rsx: Implement DECR memory layout (#7906) 2020-03-31 05:12:30 +01:00
Eladash 1510505b30 Fix sys_rwlock_wlock timeout event 2020-03-30 14:22:15 +03:00
Jan Beich afce3ee2ed Qt: add more headers for non-Vulkan
rpcs3/rpcs3qt/emu_settings.cpp:111:44: error: use of undeclared identifier 'g_cfg'
        for (const auto& v : cfg_adapter::get_cfg(g_cfg, begin, end).to_list())
                                                  ^
rpcs3/rpcs3qt/emu_settings.cpp:262:60: error: use of undeclared identifier 'Emulator'
        fs::create_path(title_id.empty() ? fs::get_config_dir() : Emulator::GetCustomConfigDir());
                                                                  ^
rpcs3/rpcs3qt/emu_settings.cpp:276:39: error: use of undeclared identifier 'Emulator'
                const std::string config_path_new = Emulator::GetCustomConfigPath(m_title_id);
                                                    ^
rpcs3/rpcs3qt/emu_settings.cpp:277:39: error: use of undeclared identifier 'Emulator'
                const std::string config_path_old = Emulator::GetCustomConfigPath(m_title_id, true);
                                                    ^
rpcs3/rpcs3qt/emu_settings.cpp:308:17: error: use of undeclared identifier 'Emulator'
                config_name = Emulator::GetCustomConfigPath(m_title_id);
                              ^
rpcs3/rpcs3qt/emu_settings.cpp:319:21: error: use of undeclared identifier 'g_cfg'
        if (config_name == g_cfg.name || m_title_id == Emu.GetTitleID())
                           ^
rpcs3/rpcs3qt/emu_settings.cpp:319:49: error: use of undeclared identifier 'Emu'
        if (config_name == g_cfg.name || m_title_id == Emu.GetTitleID())
                                                       ^
rpcs3/rpcs3qt/emu_settings.cpp:322:3: error: use of undeclared identifier 'g_cfg'
                g_cfg.from_string(config.to_string(), true);
                ^
rpcs3/rpcs3qt/emu_settings.cpp:324:8: error: use of undeclared identifier 'Emu'
                if (!Emu.IsStopped()) // Don't spam the log while emulation is stopped. The config will be logged on boot anyway.
                     ^
rpcs3/rpcs3qt/emu_settings.cpp:326:51: error: use of undeclared identifier 'g_cfg'
                        cfg_log.notice("Updated configuration:\n%s\n", g_cfg.to_string());
                                                                       ^
2020-03-30 10:52:46 +02:00
Eladash 019aae8bec sys_net: Fix access violation handling (#7901)
Should fix page faults notifications on unmapped menory on Gran Turismo 5.
2020-03-30 05:53:17 +01:00
JohnHolmesII 6d32ebecca CI: Update Azure cache task to newer version 2020-03-29 14:10:07 +03:00
JohnHolmesII ad13075b36 Build: Fix potential issue with Windows builds not receiving correct
branch info
2020-03-29 14:10:07 +03:00
JohnHolmesII 8581a2775a CI: Add workaround for exporting variables in Azure
Using '-x' to echo commands in the shell causes
the Azure process commands to be processed twice
2020-03-29 14:10:07 +03:00
Whatcookie cc100f4008 Fix alignment on embedded spu elf searching (#7894)
- They are actually 128 byte (1024 bit) aligned.
2020-03-29 03:54:45 +01:00
Nekotekina cba9ed3527 CMake: add /MT for MSVC 2020-03-28 17:21:03 +03:00
Nekotekina dcc269128f SPU LLVM: runtime multithreaded compilation
Active for CPU with 12 or more threads.
2020-03-28 17:17:51 +03:00
Nekotekina aae338a91c named_thread_group: add a default constructor 2020-03-28 17:17:51 +03:00
Nekotekina 9db088e17b SPU LLVM: log hash in some places 2020-03-28 17:17:51 +03:00
illusion c2a24fd047 readme.md: Fix Azure Badge URL (#7888)
previously leads to RPCS3.glslang instead of RPCS3.RPCS3
2020-03-28 10:38:38 +00:00
Eladash 4215499b7f rsx: Fix typo in NV4097_SET_TRANSFORM_PROGRAM range 2020-03-28 11:07:34 +03:00
unknown 049825812e RSX: Restrict analyser loop error 2020-03-28 09:42:13 +03:00
Eladash 66bd8308d9 lv2: Wait for rescheduling before confirming timeouts (#7875) 2020-03-28 03:16:59 +00:00
xddxd d96dabcd60 rsx: Rename current_instrution to current_instruction (#7883) 2020-03-28 02:46:48 +00:00
Jan Beich 777f0a7c82 Implement IsDebuggerPresent on BSDs (#7880) 2020-03-28 01:57:41 +00:00
Jan Beich 58492ef92d build/cmake: add option to use system-wide libcurl package (#7882) 2020-03-28 00:49:31 +00:00
JohnHolmesII f5a51599d9 CD: Fix experimental build warning for Travis 2020-03-27 23:00:22 +03:00
JohnHolmesII 6712ac0a72 Build: Do not warn for local builds 2020-03-27 23:00:22 +03:00
Zion d6258fce54 Change target to the correct commit SHA on RPCS3/RPCS3-binaries-win side 2020-03-27 19:21:41 +03:00
AniLeo 4e28aa4681 azure: Fix win releases repository 2020-03-27 18:47:35 +03:00
JohnHolmesII 70d6a12894 CI: Port Windows build to Azure Pipelines (#7757)
* CI: Port Windows build to Azure Pipelines from Appveyor

* CI: Split Windows build into scripts

* CI: Remove Appveyor

* CI: Add GitHub Release deployment to Azure Windows Build

* VCS: Add full branch name function to rpcs3_version

The STRINGIZE macro was a little awkward, and difficult to control
at configure time. Since other version information is already
included, the full branch name is now added as a function. It's
runtime instead of compile-time checking, but it seems worth it.

* CI: Overhaul Windows setup script

Previously, there was no way of forcing a re-download
of cached dependencies when they were replaced by new ones. In
addition, there was really no verification of downloads or cache.
Now, changing a few lines at the top of the file will automagically
force a cache update.
2020-03-27 16:37:27 +03:00
AniLeo 96185af64f .gitignore: Ignore cmake autogen libusb files
Co-Authored-By: Eladash <elad3356p@gmail.com>
2020-03-27 15:26:28 +03:00
AniLeo c00ee7ed5b hle: cellSysutilNpEula
Add missing function names and stub all functions

Co-Authored-By: Eladash <elad3356p@gmail.com>

Co-Authored-By: Eladash <elad3356p@gmail.com>
2020-03-27 15:26:28 +03:00
RipleyTom cd4eed0704 Gives ANSI path to curl CURLOPT_CAINFO 2020-03-27 14:23:20 +03:00
dio-gh 2aac46efcc Update building instructions (#7872)
* Update building instructions

Makes the link for the prebuilt LLVM point to the `_mt` version now, since the previous one will fail to compile on latest.

* Update glslang link

Replaces the outdated not-shady-at-all link with the new, shiny and correct one.

* Fixing a 'the'

Fixes a 'the'.

Co-authored-by: Ani <ani-leo@outlook.com>
2020-03-27 10:34:13 +03:00
scribam a2bf0719ea cmake: Fix windeployqt command line for release builds (#7871) 2020-03-27 00:33:07 +00:00
Ani e8177906e7 System/Load: Handle PSP Remaster (#7857)
This just makes RPCS3 execute PSP Remaster games through 
psp_emulator.self
pspemu does not work yet, so the boot will fail, but at the least it 
starts loading
2020-03-26 20:03:00 +03:00
Nekotekina 17a9ce6fb9 Fix PS1 game_path
Don't assume local path has '/dev_hdd0' in it.
2020-03-26 18:32:42 +03:00
Eladash 36d8553f9c liblv2 HLE: Fix entryx of start/stop prx functions 2020-03-26 17:52:45 +03:00
Eladash 9d971e3b07 rsx: More strict infinite desync detection
6 desyncs per second for 1.5 seconds is pretty bad already.
2020-03-26 17:52:45 +03:00
Eladash 7ed570dc4a PPU LLVM: Add relocation 5 for ADDIS
+ Add some more for u16 relocations (4, 5, 6), simplify logic.
2020-03-26 17:52:45 +03:00
Nekotekina a5ba4a18df Fix windeployqt command line for release builds 2020-03-26 16:53:21 +03:00
Nekotekina 214a3ba21b Return Appveyor to VS 16.5 2020-03-26 16:48:44 +03:00
Maxim Kulyk 30a8cadf60 [MSVC] Remove unnecessary configurations and properly fix curl 2020-03-26 15:56:40 +03:00
Nekotekina 15a36bb844 Adjust llvm hash (affects nothing) 2020-03-26 15:29:15 +03:00
Nekotekina fa29c5aa94 ppu_iname: refactor to use actual strings 2020-03-26 15:28:41 +03:00
Nekotekina 8d1a9dce91 spu_iname: refactor to use actual strings 2020-03-26 15:26:55 +03:00
Eladash 453478c98b PPU LLVM: Log unsupported relocation opcode 2020-03-26 15:22:45 +03:00
Eladash ccc7cb7994 Log IDs of loaded prx modules 2020-03-26 15:22:45 +03:00
Eladash 38c8dd98b4 rsx: Implement basic infinite FIFO desync detection 2020-03-26 15:22:45 +03:00
Maxim Kulyk 433a21286a [appveyor] Update glslang and LLVM links 2020-03-26 15:21:53 +03:00
Maxim Kulyk ec4287cbd3 static RT 2020-03-26 15:21:53 +03:00
Maxim Kulyk e58fa7d51f Fix curl 2020-03-26 15:21:53 +03:00
Maxim Kulyk 70f1302fa7 [MSVC] Disable exceptions and spectre mitigation in common_default.props
Disabling exceptions in common_default.props ensures that all projects do not use them since not all projects import rpcs3_default.props.

Spectre mitigation is disabled by default unless WDK is installed so this is more of a just in case measure since rpcs3 does not need it.
2020-03-26 15:21:53 +03:00
Maxim Kulyk 3b9f419859 [MSVC] Added common props to llvm and glslang
common_default.props are now imported when building LLVM and glslang via MSVC.
2020-03-26 15:21:53 +03:00
Maxim Kulyk d26c465911 [MSVC] Move libcurl and wolfssl project files
libcurl and wolfssl were moved to rpcs3 source control to make buildsystem changes easier.

common_default.props and common_default_macros.props included to project files.
Int and Out Dirs changed to default:
<OutDir>$(SolutionDir)lib\$(Configuration)-$(Platform)\</OutDir>   <IntDir>$(SolutionDir)tmp\$(ProjectName)-$(Configuration)-$(Platform)\</IntDir>
2020-03-26 15:21:53 +03:00
Eladash 158e34faca rsx: Reset all method registers at rsx_state::init() 2020-03-25 17:51:59 +03:00
Eladash 768b4f8c65 rsx: Improve NV308A_COLOR
* Fix NV308A_COLOR methods range.
* Batch NV308A_COLOR methods execution together.
* Fix termination of bind_range<> in rsx methods binding.
2020-03-25 17:51:59 +03:00
Eladash 150d1bcdd5 sys_prx_unload_module: Implement CELL_PRX_ERROR_NOT_REMOVABLE 2020-03-25 16:22:47 +03:00
Eladash e1f2f3f081 sys_prx: Improve sys_prx_start/stop_module
* Add missing error codes, "internal" errors are ones which are not reachable from liblv2.sprx functions

* Implement SYS_PRX_NO_RESIDENT (unloading module) for _sys_prx_start_module.
2020-03-25 16:22:47 +03:00
Eladash 717dd1625c sys_prx: Implement sys_prx_get_module_list 2020-03-25 16:22:47 +03:00
Megamouse a11c77c009 Qt: fix mem leaks in screenshot and save managers 2020-03-25 11:50:06 +01:00
Megamouse 844f9683ec Qt: add naive lazy loading to screenshot manager 2020-03-25 11:50:06 +01:00
Megamouse f27de28ee9 Qt: add open file location to screenshot preview
Remove duplicate slash from screenshot path
2020-03-25 11:50:06 +01:00
Megamouse bd49ad358c Qt: move open_dir to qt_utils 2020-03-25 11:50:06 +01:00
Nekotekina 89514c043a Fix "Unknown option: updating" 2020-03-25 11:23:38 +03:00
Nekotekina b33648fd14 Implement SAFE_BUFFERS as __attribute__((no_stack_protector))
It was doing nothing outside of MSVC. Still seems doing nothing.
2020-03-25 11:18:48 +03:00
Nekotekina 471db3219d Finalize constexpr ppu_decoder<> thing
Move SSSE3 checks to runtime in PPUInterpreter.cpp
2020-03-25 11:18:48 +03:00
Megamouse fd3522436a overlays/osk: add more panels 2020-03-25 03:54:49 +01:00
Nekotekina 1ceb779a38 Make ppu_decoder<> objects constexpr (partial) 2020-03-24 13:46:46 +03:00
Nekotekina ecb6d38451 Roll back to VS 16.4 on Appveyor
Workaround VCRUNTIME140_1.dll dependency.
2020-03-24 12:28:20 +03:00
Nekotekina a936533eb1 Make spu_decoder<> objects constexpr 2020-03-24 12:18:37 +03:00
Eladash eb1de04ca8 _sys_lwmutex_lock: log new case of ESRCH 2020-03-23 23:18:21 +03:00
Eladash 2aff36647a Make sceNpDrmProcessExitSpawn(2) execute sys_game calls correctly 2020-03-23 23:18:21 +03:00
Nekotekina 19e20d9c19 Auto-Updater: increase lock file waiting timeout in the case of updating
Normal case: timeout reduced from 3s to 2s.
Updating case: increased timeout to 10s.
2020-03-23 22:52:05 +03:00
Nekotekina 3d78694590 Debug: measure initialization time (before main() function) 2020-03-23 22:18:45 +03:00
Eladash 52360b3f98 liblv2 HLE: Fix sys_lwmutex_lock assertation 2020-03-23 21:37:37 +03:00
Eladash 08e66ab14c Minor warning fixes 2020-03-23 21:37:37 +03:00
Eladash 3de41bfea7 _sys_lwcond_signal: Make mode 3 respect ordering of the sleep queue 2020-03-23 21:37:37 +03:00
Megamouse 1537f505a5 cellGem: fix move_handler::mouse left click 2020-03-23 15:10:01 +03:00
Nekotekina 5ebc538d7e Workaround for VS 16.5
Strange codegen bug didn't promote s32 to u64.
2020-03-23 14:48:49 +03:00
kd-11 4965bf7d7a gl/vk: Refactor draw call handling and stub shader interpreter
- Refactors backend draw call management to make it easier to extend
  functionality.
- Stubs shader interpreter functionality.
2020-03-23 14:47:28 +03:00
illusion a0509328d4 [README.md] Update to Visual C++ Redistributable 2019 2020-03-23 12:23:18 +03:00
Eladash 537f175f52 Another fixup after #7826 2020-03-23 11:54:19 +03:00
Eladash 017ef5a94e Minor fixup after #7826 2020-03-23 10:59:51 +03:00
Eladash cccc32fa9d sys_lwmutex/lwcond: track lwcond waiters (#7826)
In lwmutex destroy syscall, wait for pending waiters.
2020-03-23 10:30:17 +03:00
sL1pKn07 9de9ec1f01 Fix build with Qt 5.15+ 2020-03-23 07:23:56 +03:00
Megamouse ef10ed4499 Qt: Add basic screenshot manager 2020-03-22 23:40:55 +01:00
Megamouse b447e6f55d Qt: use simple curl wrapper to avoid some pitfalls 2020-03-22 19:16:25 +01:00
Megamouse 3c63db93ed Qt: fix double slash in updater tmp_folder 2020-03-22 19:16:25 +01:00
Megamouse da09badd8d Qt: simplify current_build in update manager 2020-03-22 19:16:25 +01:00
Megamouse 7f8d802bd5 Qt: fix log message in update manager 2020-03-22 19:16:25 +01:00
Megamouse 532215fb81 Qt: show welcome dialog before showing the app
Fixes interference with update manager
2020-03-22 19:16:25 +01:00
Megamouse 13e166084d Qt: use Localized::GetVerboseTimeByMs 2020-03-22 19:16:25 +01:00
Emmanuel Gil Peyrot 425e032a62 SPU: Copy with memcpy() instead of hand-rolled SSE2
In some very unscientific benchmark:
spu_thread::do_dma_transfer() was taking 2.27% of my CPU before, now
0.07%, while __memmove_avx_unaligned_erms() was taking 1.47% and now
2.88%, which added makes about 0.8% saved.
2020-03-22 15:51:03 +03:00
Nekotekina 5261886449 CURL_STATICLIB macro cleanup
Also move includes from headers. CURL is just void.
2020-03-22 14:13:52 +03:00
Nekotekina e606130262 Memoize and print r3-r6 under Current function in the ppu_thread::dump() 2020-03-22 14:13:52 +03:00
Megamouse 7d33ca7059 Qt: use QDateTime in update manager 2020-03-22 14:13:33 +03:00
RipleyTom af4efafae1 Remove Qt5Network Qt5OpenGL and Qt5QML dependencies 2020-03-22 13:48:43 +03:00
Megamouse 09a8974786 Qt: fix curl threads 2020-03-22 13:48:43 +03:00
RipleyTom b1d8bf754e Replace QNetwork operations with libcurl + wolfssl 2020-03-22 13:48:43 +03:00
Eladash 132c3e1c1a kernel explorer: Add information about memory containers 2020-03-22 12:41:02 +03:00
Eladash dc839e1784 lv2: Do not lose r3 data on syscalls
Allows to get the ID of the lv2 sync objects in the debugger by looking at r3's content.
2020-03-22 12:41:02 +03:00
Eladash f1cf62ac57 kernel explorer: Implement ability to view lwmutex owner 2020-03-22 12:41:02 +03:00
kd-11 12044bd8b0 rsx: Properly calculate vertex range when divisor is active
- The upper bound is to be rounded up, not down.
2020-03-22 10:57:47 +03:00
Eladash 01cafc042d cellSaveData: Ensure savedata_context members are 16-byte aligned
I saw stvx v128{0} (aligned 16-bytes store) usage on the first 16-bytes of CellSaveDataCBResult before funcStat in fw.
Also I saw 4 stw of u32{0} on it as well before funcFile, funcFixed and funcList.
So just add the resets for result before all callbacks, and make all
members of savedata_ontext 16 -byte aligned in case there are more
members guaranteed to be aligned.
2020-03-21 19:05:20 +03:00
Eladash ceaee0ec68 cellSaveData: Clear traces of setList setup from setBuf->buf, add missing memset
* Always memset 0 setBuf->buf (to bufSize) before funcStat if the direcory is not new.
* Always memset 0 setBuf->buf (to bufSize) if listGet->dirNum became non-zero (listGet->dirListNum can be zero yet memset still occurs) .
* Clear traces of setList setup before funcStat (after funcFixed/List, only if listGet->dirNum != 0, callback can hack this value and prevent the memset).
2020-03-21 19:05:20 +03:00
Eladash b6a288d383 cellSaveData: Skip directory items in savedata_get_list_item 2020-03-21 19:05:20 +03:00
Eladash ae14eb0747 Minor fix of cellUserInfoGetStat
stat == nullptr is allowed, fix invalid id error code.
2020-03-21 19:05:20 +03:00
Eladash be0e586671 cellSaveData: Add error checks for cellSaveData(User)GetListItem 2020-03-21 19:05:20 +03:00
Eladash 66916df4ae cellSaveData: Set listSet->focusPosition to LISTHEAD by default 2020-03-21 19:05:20 +03:00
Eladash fae46bf194 cellSaveData: Add CELL_SAVEDATA_ERROR_NOUSER 2020-03-21 19:05:20 +03:00
Eladash e1cb827488 PPU Precise: Fix FMADDS, FMSUBS, FNMSUBS, FNMADDS 2020-03-21 16:32:09 +03:00
Eladash 3a36b713ce Dont spam syscalls stats if emu is paused 2020-03-21 16:31:18 +03:00
Nekotekina 49d8731c1c Thread.h: fix warning 2020-03-21 13:49:41 +03:00
Nekotekina ad552c6b78 Appveyor: use previous VS 2019 image (workaround) 2020-03-21 13:20:49 +03:00
Nekotekina ebb007ecb4 Update llvm hash 2020-03-21 12:38:16 +03:00
Eladash 9acf8e283d Fix OSK thread exit condition 2020-03-21 12:37:29 +03:00
Nekotekina 7f5dd1dd62 Fix thread_base::join 2020-03-21 10:36:04 +03:00
Nekotekina c577bd2111 Implement thread_state::errored
State after calling thread emergency_exit() function.
Also default-construct thread result in this case.
2020-03-20 21:31:27 +03:00
Megamouse eb2dcaf602 Qt: fix some translation bubus 2020-03-20 01:43:08 +01:00
Megamouse fd8cda0f2b overlays/osk: fix selection after changing panels
We now try to keep the current x and y selected after panel changes.
Also change some copy to ref
2020-03-19 21:10:08 +01:00
Megamouse c63f77e3b0 overlays/osk: fix full width characters 2020-03-19 21:10:08 +01:00
Megamouse a1f70bf96e overlays/osk: do not change the preview text on empty input
This prevents that the placeholder disappears
2020-03-19 21:10:08 +01:00
Megamouse f1127f1894 overlays: implement osk panels 2020-03-19 21:10:08 +01:00
Zion Nimchuk 27367dc493 Fix clang warnings 2020-03-19 19:20:27 +03:00
Zion Nimchuk 5ba07ed50f Partially revert 6ec149d. Fixes #7799
Turns out linking with LTO still doesn't play nice with Qt.
For reference:
https://github.com/InBetweenNames/gentooLTO/issues/444
https://bugreports.qt.io/browse/QTBUG-61710
2020-03-19 19:20:27 +03:00
Eladash 1dbb5422a2 Avoid a segfault when reading ppu stack contents in debuggers
TODO: lock vm mutex.
2020-03-19 14:18:05 +03:00
Eladash a3289e9d40 Fix memory leak in rsx debugger 2020-03-19 14:18:05 +03:00
Eladash f2d6a1ff60 disasm: Improve instructions spacing 2020-03-19 14:18:05 +03:00
Eladash fd45bf5fba debugger: Force aligned memory view
Fixes a corner case viewing unaligned memory at the end of spu memory.
Also unaligned view isn't suitable for the debugger, for these purposes the memory viewer should be used instead.
2020-03-19 14:18:05 +03:00
Eladash e3668cc26c Fix a segfault in memory viewer
Also a memory leak.
2020-03-19 14:18:05 +03:00
Eladash 7139c4fbab Fix bug introduced by #7797 2020-03-19 07:22:05 +03:00
Malcolm Jestadt 0bfdc1f62e PPU LLVM: Improve VMADDFP and VNMSUBFP
- Use native FMA to emulate VMADDFP, with a fallback for processors that don't support FMA
- Use native FMA to emulate VNMSUBFP as well, but note that it differs from the emulated path with regards to negative zero
2020-03-19 06:47:16 +03:00
Eladash b11e8f8b8d Minor fix of sys_ppu_thread_yield return value
Account for a special case where threads were rotated but no context
switch was made.
2020-03-19 06:45:14 +03:00
Eladash 7be35315da Fix lv2 sys_lwcond/sys_lwmutex kernel explorer names 2020-03-19 06:45:14 +03:00
Nekotekina aa5c6c4d2b Cleanup std::is_pod usage (deprecated in C++20) 2020-03-18 18:28:46 +03:00
Nekotekina 6a2571d0e1 SPU: print current chunk hash in dump 2020-03-18 18:28:46 +03:00
Nekotekina 20f7544a8a SPU profiler: minor change
Use std::greater to sort
2020-03-18 18:28:46 +03:00
Zion Nimchuk 6ec149d8f5 Build rpcs3's AppImages with link time optimization
On the clang build we now link with lld, and on GCC we now link with Gold
2020-03-18 13:07:44 +03:00
Zion Nimchuk 2678ac6382 Change default audio backend on non-windows
FAudio is preferred, but if we didn't build with FAudio, we default to OpenAL
Also changed it to explicitly use xaudio backend on windows, rather than the static cast we had before.
2020-03-18 09:38:13 +01:00
Eladash c9b5ba4a5c BEType.h: use common initial sequance in v128
Partially obey 'strict type aliasing' rule.
2020-03-17 18:22:13 +03:00
Eladash 03a6d67c6c Log sys_lwmutex/sys_lwcond names as strings
Use std::string_view instead of creating a temporary NTS string when reading object name.
2020-03-17 18:22:13 +03:00
Eladash a9f492b605 sys_spu: Fix oops in sys_raw_spu_destroy after #7782
'id' is not the idm id, also explicitly join the thread so a situation where the thread is still active and communicating other threads (e.g via MMIO or MFC) yet its ID is removed won't happen. (logic breakage, destroyed thread can't be  active)
2020-03-17 18:22:13 +03:00
Eladash 664d606123 Add return value of sys_ppu_thread_yield 2020-03-17 18:22:13 +03:00
Eladash 00d25a191b lv2: Uncomment sys_ppu_thread_stop/restart 2020-03-17 18:22:13 +03:00
Eladash 3566faabd9 Add missing lv2_obj::sleep when joining interrupt thread 2020-03-16 21:06:33 +03:00
Eladash 7e224c5585 cellVdecClose: Followup fix to #7663
Avoid executing sys_interrupt_thread_disestablish on different ppu thread with the same ID.
2020-03-16 21:06:33 +03:00
Eladash 9e14e835e7 Fix sys_raw_spu_destroy 2020-03-16 21:06:33 +03:00
Megamouse 33d01fd252 log: properly escape all html except newlines 2020-03-15 20:41:24 +03:00
kd-11 d25ba03e82 vk: Lazy evaluate renderpass scope
- Spamming the driver with renderpass open/close cycles is bad for performance.
2020-03-15 18:39:40 +03:00
kd-11 7025985c0d rsx: Improve section scanning when updating surface cache resources in blit engine. 2020-03-15 16:51:23 +03:00
kd-11 a756c0679e rsx: Implement cross-aspect slice gathering
- Fixes a data leak that can happen when a surface is rejected due to aspect mismatch.
- Mismatch can lead to rejection due to area covered excluding the RTT and inevitable upload a texture from CPU at the same location.
- Overlapping fbo/shader_read resources are not allowed.
2020-03-15 16:51:23 +03:00
Eladash 377e06a4a2 rsx: Fix unknown Blend equation 2020-03-15 09:53:15 +03:00
Nekotekina a0612ff4a7 Disable failing OSX build on travis
Since we have Azure maybe someone will have better luck with it.
2020-03-14 23:07:40 +03:00
Nekotekina 11d5bd3452 Try to re-enable Azure Pipelines for PRs 2020-03-14 22:50:17 +03:00
Rose 231e837f9b [UI] Grey out AA and Aniso settings under strict rendering (#7773)
* Grey out AA and aniso under strict rendering

* Сhange aniso UI string to 'Auto'

Co-authored-by: Ivan <Nekotekina@users.noreply.github.com>
2020-03-14 20:45:41 +03:00
Nekotekina 45389dca51 PPU: minor fix for ppu_join_status::max
Don't treat it as special "invalid" value.
2020-03-14 20:36:56 +03:00
Eladash 4e6934c9dc Minor idm::remove_verify usage optimization in sys_interrupty.cpp
Avoid copying a shared ptr, create a weak ptr instead.
2020-03-14 19:26:37 +02:00
Eladash 74c05ec5ee PPU Disasm: Fix non-link extended BCCTR forms 2020-03-14 19:26:22 +02:00
Eladash cb4192bce9 vm: Log all guest memory bases at startup 2020-03-14 18:30:14 +02:00
Eladash 83a2204f87 PPU Disasm: Fixup BCCTR after #7775 2020-03-14 18:30:14 +02:00
Eladash c16124f0d9 idm: Fix minor race in cellVdecClose, sys_raw_spu_destroy...
Because of racing removal of IDs vs the shared pointer owned object
2020-03-14 18:30:14 +02:00
Eladash efe6e1eb0a sys_ppu_thread: Make PPU id removal after exit atomic with descheduling
* Make PPU id removal after exit atomic with descheduling
* Make joining thread scheduling atomic with thread exit sleep.
* Update sys_ppu_thread_stop/restart.
* Add idm::remove_verify.
2020-03-14 18:30:14 +02:00
Eladash c3d36940c7 cellSaveData: do not allow to read/write/delete system files in funcFile
Return param error 70 as realhw in this case.
2020-03-14 16:12:18 +03:00
Eladash ff341fe597 PPU Disasm: Fix branches spacing
Null terminator was added at the end which prevented proper spacing.
2020-03-14 16:12:18 +03:00
Eladash db71d4852a cellSaveData: Fix adding file entries to PARAM.SFO on error 2020-03-14 16:12:18 +03:00
Eladash 88ee198d78 cellSaveData: return ERROR_FAILURE on funcFile deletion failures 2020-03-14 16:12:18 +03:00
Nekotekina 9176ca084c VFS: clarify escape/unescape cannot work on paths
With recent changes, it can only work with file or directory name.
2020-03-14 16:01:55 +03:00
Nekotekina 70cd8afafa VFS: improve vfs::escape (escape NUL, LPT, COM...)
Windows legacy trash.
2020-03-14 15:07:59 +03:00
Nekotekina 6a8f5e6b38 VFS: fix vfs::get
Escape each path element separately.
2020-03-14 14:25:37 +03:00
Nekotekina 6c234b48ce VFS: Fix possible out of range in vfs::escape 2020-03-14 13:33:25 +03:00
Nekotekina bf957a575c VFS: fix out of range problem in vfs::unescape
Also improve unescape with "!" allowing any character
2020-03-14 13:15:46 +03:00
Nekotekina 0c65a1721d VFS: escape multidot names like ... 2020-03-14 12:34:28 +03:00
Eladash 5d78d81c00 cellSaveData: Filter directory lists to allowed savedata directories
Filters "." and "..", as well as possible wrong directories added by the user.
2020-03-13 22:43:27 +03:00
Eladash b21b4faca8 cellSaveData: Fixed savedata lock after fmt::throw_exception 2020-03-13 22:43:27 +03:00
Eladash d58f52ff31 cellSaveData: Add some listSet error checks
* Check listSet->fixedListNum.
* Check listSet->fixedList for nullptr and its directory items names.
* Check listSet->focusDirName for nullptr and directory name.
* Check listSet->newData->iconPosition.
* Check listSet->newData->dirName for nullptr and directory string.
* Check statSet->setParam->parental_level for old sdk.
* Return an error if listSet->focusPosition is NEWDATA and listSet->newData is nullptr.
* Simplify savedata directory list selection.
2020-03-13 22:43:27 +03:00
Eladash 54af8ec544 cellSaveData: funcFile fixes
* Allow '_' at filenames start and extension.
* Check if reading offset is valid, fix error code to CELL_SAVEDATA_ERROR_FAILURE.
* Don't create empty file on error of write ops.
* Don't allow "." and ".." filenames on funcFile, return CELL_SAVEDATA_ERROR_BROKEN.
2020-03-13 22:43:27 +03:00
Eladash fdf47f43d8 cellSaveData: refactor param error 70 checks
Also extend the check to check empty name.
2020-03-13 22:43:27 +03:00
kd-11 2ae83782e1 vk: Fix potential MTRSX deadlock in case of a race condition 2020-03-13 22:06:04 +03:00
Eladash f3877d11e8 rsx: Fix initial boolean state of m_textures_dirty and m_vertex_textures_dirty 2020-03-12 21:36:43 +01:00
Eladash 28e9cade2c GUI/rsx capture: Disable capturing if no game is running! 2020-03-12 21:36:43 +01:00
Eladash c04abac630 rsx capture: Fix exceptions handler, fix tiny race condition on capture new capture 2020-03-12 21:36:43 +01:00
kd-11 7e9dbeff7b vk: Fix MTRSX deadlock (#7766) 2020-03-12 22:29:58 +03:00
Megamouse f0edcc16fe evdev: add more buttons to button list 2020-03-12 19:43:52 +01:00
Megamouse e7adef9fe1 evdev: remove unused call to update_device 2020-03-12 19:43:52 +01:00
Megamouse 9c5da55dca evdev: Some random cleanup 2020-03-12 19:43:52 +01:00
Nekotekina d802b3d8b2 Remove -fno-strict-aliasing
It was added due to miscommunication.
2020-03-12 16:04:57 +03:00
Nekotekina 04dedb17eb Disable exception handling.
Use -fno-exceptions in cmake.
On MSVC, enable _HAS_EXCEPTION=0.
Cleanup throw/catch from the source.
Create yaml.cpp enclave because it needs exception to work.
Disable thread_local optimizations in logs.cpp (TODO).
Implement cpu_counter for cpu_threads (moved globals).
2020-03-12 16:03:08 +03:00
kd-11 47bbfdd2aa vk: Change texture cache memory management for disposed textures
- Use global resource manager instead of using the 2-frame hold behavior.
- Fixes high VRAM usage in some games
2020-03-11 16:29:34 +03:00
Nekotekina 6bd96a4590 Fix thread_base::finalize (and emergency_exit, collaterally)
Forgot to reset futex callback. Could cause crashes.
2020-03-10 23:23:32 +03:00
Nekotekina 92eeec39b7 Improve Stop Watchdog
Make it less possible to interfere with the debugger.
2020-03-10 23:00:23 +03:00
Nekotekina 656db6c668 Fatal errors: concatenate multiple args after --error
It should fix error dialogs on Windows since it decomposes the arg string.
2020-03-10 22:42:33 +03:00
kd-11 7989de9d16 vk: Properly release dma resources. 2020-03-10 22:02:02 +03:00
Megamouse 3ea94c286b input/overlays: fix premature pad interception removal
shader compilation and trophy notifications shouldn't cancel the pad interception during proper dialogs
2020-03-10 19:04:32 +01:00
Bird Egop 4e25daffa6 Explicitly rename has_512 into has_avx512 (#7751) 2020-03-10 19:21:00 +03:00
Nekotekina b4f416cb76 Fix narrow warning in ds4_pad_handler.cpp 2020-03-10 18:45:49 +03:00
Nekotekina 1678b37aa0 Use TRAP on segfault with debugger (Linux) 2020-03-10 14:06:06 +03:00
Nekotekina adfd8ab43c Break in the debugger in thread_ctrl::emergency_exit
Implement IsDebuggerPresent analog for non-Windows systems.
2020-03-10 13:28:24 +03:00
Nekotekina 87d4b14ca9 Pause only on fatal messages
Also make some access violation an error since we don't pause on it.
2020-03-10 11:26:42 +03:00
Nekotekina d3eb267ba9 Logs: add 'always' method for debugging 2020-03-10 11:23:56 +03:00
kd-11 12b73c8bdc rsx: Fix copypasta 2020-03-09 17:20:24 +03:00
Eladash 5751b77688 GUI: followup to #7347
Show "Reboot" on current running game when there's no config.
2020-03-09 16:07:14 +03:00
Eladash 636ed4a48b HLE cellGcmSys: Avoid calling sys_rsx syscalls in rsx code 2020-03-09 16:07:14 +03:00
Eladash af7cdcb5c7 Add forgotten error check in sys_spu_thread_group_connect_event 2020-03-09 16:07:14 +03:00
kd-11 2985a39d2e rsx: Rewrite async decompiler 2020-03-09 14:59:25 +03:00
Nekotekina 609c0d46af Implement stop watchdog
Shows fatal error if stopping takes more than 5s.
2020-03-09 13:20:49 +03:00
MSuih a2b6546d37 Fix framelimit/aspect ratio width 2020-03-08 21:56:48 +01:00
Nekotekina 9dca2887d8 Fixup for Emu.Pause()
Remove some reduntant calls.
Don't pause on unknown sys_fs_fcntl operation.
2020-03-08 22:03:16 +03:00
Nekotekina 6268a2d384 Improve report_fatal_error()
Previously it could cause secondary segfault on Linux.
2020-03-08 22:03:15 +03:00
Nekotekina c87beaa694 Use _wexecl on Windows
Allows original path to contain any Unicode character.
2020-03-08 20:45:34 +03:00
Nekotekina 1bbe2e9b15 Simplify report_fatal_error
Those semaphores didn't achieve anything.
Launch separate process if Qt is already initialized.
2020-03-08 20:45:34 +03:00
Nekotekina e40019354c Pause emulation on any fatal log message. 2020-03-08 20:45:34 +03:00
Nekotekina 07e1766a7c Implement thread_ctrl::emergency_exit()
Replace exception throws with this.
2020-03-08 15:11:02 +03:00
illusion0001 814c73407d overlay: set minimum update interval to 1ms 2020-03-08 15:05:42 +03:00
Eladash 5692c3de04 Fix sceNpUtilCmpNpId 2020-03-08 12:49:34 +03:00
kd-11 8214425a3c rsx: Fix framebuffer native layout for X32_FLOAT
- It was not matching the order laid out for normal textures uploaded from CPU.
2020-03-08 11:43:49 +03:00
kd-11 84a542fbce rsx: Blit engine improvements
- Detect writes to the display output memory and handle it specially.
  It already defines a known 2D region.
- Try and detect situations where raw transfers would be of benefit.
2020-03-08 10:30:13 +03:00
Megamouse ab4189998c Qt: don't create stupid default.ini file when resetting gui configs 2020-03-08 00:06:48 +01:00
Megamouse 9b672cb969 Qt: Improve tooltip areas in network tab 2020-03-08 00:06:48 +01:00
Megamouse 5f247cbedc Qt: Backup current gui config before applying another one
Also fixes some strange issues caused by the pointer
2020-03-08 00:06:48 +01:00
Megamouse 53676067fc Qt: remove gui settings default shenanigans 2020-03-08 00:06:48 +01:00
Megamouse 11bc7de0ca Qt: more code cleanup in gui files 2020-03-08 00:06:48 +01:00
Megamouse 091dcc1052 Qt: fix play button state when booting rpcs3 for the first time 2020-03-08 00:06:48 +01:00
Megamouse 934a2eb9fa Qt: some code cleanup in gui files 2020-03-08 00:06:48 +01:00
Megamouse 426643c44d Qt: Prefer currently selected game when pressing the play button
Also rename Start to Play
2020-03-08 00:06:48 +01:00
Megamouse 7dd36ff829 Qt: Fix CurrentSelectionIconPath for game grid
Fixes deselection issue when booting a game in the game grid
2020-03-08 00:06:48 +01:00
Megamouse 0c45457101 Qt: Add title and title id to button tooltips 2020-03-08 00:06:48 +01:00
Megamouse e56b3256b0 Qt: Add missing boot error dialog 2020-03-08 00:06:48 +01:00
Eladash 892f74d762 rsx: Improve frame-limiter (#7723)
* rsx: Improve frame-limiter accuracy

* lv2: Improve lv2_obj::wait_timeout response time for aborting threads

* rsx: Make stretch to display area setting dynamic

* rsx: Redefine 'auto' frame limiter to obey vblank rate

* rsx: Make frame limiter setting dynamic

* rsx: Make frame-limiter compatible with dynamic changes
2020-03-08 01:11:35 +03:00
Eladash deb6bd3e25 Atomic sys_prx_load_module_list error checks 2020-03-07 22:03:38 +03:00
Eladash 5e86ef2371 rsx: Make force cpu blit setting dynamic 2020-03-07 21:17:26 +03:00
Nekotekina e74d3c2674 Fix llvm_build.vcxproj instructions 2020-03-07 18:34:04 +03:00
kd-11 149d550f7e gl: Restore commented out line
- Byte order step was disabled for debugging and not restored
2020-03-07 17:23:25 +03:00
kd-11 70f2577b9e vk/gl: Use best-fit semantics when scanning texture cache for flippable images
- Allows sourcing flip data from the blit engine resources which avoids expensive flush and re-upload
2020-03-07 16:58:35 +03:00
kd-11 93295f7f50 vk: Fix image properties for flip temporary images to be samplable.
- In case of gamma correction or other effects, they may require shader access.
- BGRA8_UNORM is usually safe to use directly without staging memory.
2020-03-07 16:58:35 +03:00
kd-11 1725f7a34b rsx: Add anaglyph 3D filter 2020-03-07 16:58:35 +03:00
kd-11 6e3406b3f5 video: Allow selection of 3D stereo resolutions 2020-03-07 16:58:35 +03:00
Nekotekina 19107b2de5 logs.cpp: fix log format for backward compatibility
Don't add prefix for first messages.
2020-03-07 15:33:07 +03:00
Nekotekina 7599c66639 logs.cpp: print some errors if failed to create logs 2020-03-07 14:38:19 +03:00
Nekotekina b726aa5a3e logs.hpp: minor optimization for non-formatting logs
Also use single template with CharT to match fmt::format.
2020-03-07 13:52:51 +03:00
Nekotekina 12a3cdf0e8 Move Log.cpp to util/logs.cpp
Minor cleanup
2020-03-07 13:31:10 +03:00
Nekotekina e4a81b1d13 Move Log.h to util/logs.hpp 2020-03-07 12:29:23 +03:00
Nekotekina a166d3680e Don't throw on invalid whence (return fs::error::einval) 2020-03-07 11:52:54 +03:00
Nekotekina 8461a5cbe2 Add fs::error::unknown, don't throw 2020-03-07 11:22:04 +03:00
Nekotekina 66b0b78055 Logs.cpp: more code moved to main.cpp 2020-03-07 11:15:44 +03:00
Nekotekina 2209be5216 Logs: remove mem-mapped buffer and move instance lock to main.cpp
Part of the work to untangle utilities from RPCS3-specific things.
2020-03-07 10:49:09 +03:00
Eladash 1669e95870 Implement lv2_file::open()
Return accurate error codes in prx_load_module, sys_spu_image_open and overlay_load_module.
2020-03-06 21:16:46 +03:00
Nekotekina acd50eefaf Try to use designated initializers 2020-03-06 09:42:49 +03:00
Nekotekina 7514e53385 Fix SPRX/firmware installation (use a mutex) 2020-03-06 08:39:24 +03:00
Megamouse e1b4cf1557 Qt: Fix led dialog layout and use hidpi painting 2020-03-05 22:37:48 +01:00
Megamouse 9e449db0c2 Qt/Input: piggyback on existing callback for battery_level
removes ds4 timer workaround
2020-03-05 22:37:48 +01:00
Adiost f776910966 Qt/Input: new ds4 LED settings 2020-03-05 22:37:48 +01:00
Megamouse 0361184930 Fix llvm link 2020-03-05 20:33:44 +03:00
Eladash f6cf36f6a7 Fix _sys_prx_get_module_info p0pt->filename writing with 0 size 2020-03-05 18:28:56 +03:00
Nekotekina 7a8772dafa Replace std::string::npos with umax 2020-03-05 14:05:23 +03:00
Nekotekina 0a41999818 PPU LLVM: fix regression from warning fixes
Forgot that negative power is used here.
2020-03-05 11:07:40 +03:00
Megamouse f1147f70f4 Fix 7z for Debug and Release 2020-03-04 23:58:16 +01:00
Eladash 39eafc1f3b Fix Crypto/utils.cpp: extract_file_name 2020-03-05 00:31:43 +03:00
Megamouse 21b6495aaa Fix ui and sys_net warnings 2020-03-04 22:28:05 +01:00
Nekotekina Aux1 250736ece5 Fix warnings in emucore 2020-03-04 21:23:34 +03:00
Nekotekina Aux1 f2f3321952 Fix warnings in VKGSRender 2020-03-04 21:23:34 +03:00
Nekotekina Aux1 c3f3451269 Fix warnings in GLGSRender 2020-03-04 21:23:34 +03:00
kd-11 54775d91dc rsx/blit-engine: Account for a rare corner case
- It is possible to have a RTV<->DSV transfer with compatible-sized formats.
  Mark the depth size as typeless in such a situation to avoid crossing the aspect barrier with the API.
2020-03-04 21:21:59 +03:00
RipleyTom f1f5c91386 Fake PSN (#7516) 2020-03-04 13:55:35 +00:00
Eladash bb1b4bac9b Update thread_base::notify_abort() 2020-03-04 14:39:41 +03:00
Nekotekina bdbc7b5f1d PPU: use named_thread_group to compile modules
Improves internal logic by not using too many threads.
2020-03-04 14:10:38 +03:00
Nekotekina 8d847d6f1c Thread: internal cleanup
Use different, simpler algorithm in wait_for.
Although the very idea of such notifications was rotten.
2020-03-03 20:26:37 +03:00
Nekotekina d594490329 Thread: removed unused wait() with predicate.
It doesn't work this way anyway.
2020-03-03 18:33:02 +03:00
Nekotekina 6c66153372 Threads: move linux m_timer to static thread_local variable
Allows lazy allocation of the timer handle.
2020-03-03 18:33:02 +03:00
Nekotekina 5b0476e772 Update LLVM to new llvm-mirror (LLVM 11)
Use clang-cl to build LLVM on Windows.
2020-03-03 18:33:02 +03:00
Nekotekina 68f50c7035 Fix ppu_syscall_usage thread waiting
Fixed 10s hang on exit
2020-03-03 18:33:02 +03:00
Eladash 5c3d417b35 Fix cellGameDataCheck's game data creation with PARAM.SFO set/get 2020-03-03 18:31:43 +03:00
Nekotekina 01db83bc36 SPU LLVM: Rewrite fma32x4 to match FM and older asmjit stuff 2020-03-03 11:31:20 +03:00
Eladash de1774d8f2 cellSaveData: fix doneGet->sizeKB (#7674)
* cellSaveData: fix doneGet->sizeKB

* [⚠️] Warning: beware of typos [⚠️]
2020-03-03 11:24:49 +03:00
Zion Nimchuk 0f1fd059fc Properly setup Azure Pipelines using current system
Also sets up Azure artifacts (including for PRs) for AppImages
2020-03-03 10:39:30 +03:00
JohnHolmesII 87c8d1800b Azure (#7673)
* Set up CI with Azure Pipelines
2020-03-03 01:55:05 +03:00
Nekotekina 3105b21909 Print PPU Syscall Usage Stats
* Every 10 seconds
* On normal exit
2020-03-02 20:48:20 +03:00
MSuih d94b875187 Minor cleanup
- Remove log prefix as auto-updater no longer uses general channel for logging
- Swap some C-style nulls for nullpointers
- Other misc changes
2020-03-02 12:31:59 +03:00
MSuih 7129902b25 Add time logging for updater
Might help diagnose issues
2020-03-02 12:31:59 +03:00
Nekotekina bd234a7668 sys_ppu: another fixup 2020-03-02 10:35:54 +03:00
Megamouse 73a9946212 Qt: remove game window title size restriction
- Elide game window title label
- Add tooltip with format and resolved title
- Remove max length (do not wrap text to show how ridiculous it will look if it's too long)
2020-03-01 22:00:57 +01:00
Nekotekina c7fa4e2375 Fixup 2020-03-01 22:40:48 +03:00
Nekotekina 6ee9153329 sys_ppu_thread: fixing detached threads 2020-03-01 22:29:03 +03:00
InvoxiPlayGames 752c4a7b0d Fix duplicate inputs for GHLtar strumming 2020-03-01 20:10:46 +03:00
kd-11 582c98359e videoOut: Resolve 'auto' aspect ratio
- 'Auto' is not an actual aspect ratio, just a dont-care flag to set the
  display to the best fit aspect ratio.
  Decay this option into a proper ratio for cellVideoOutGetState to
return legal values.
2020-03-01 20:10:11 +03:00
Eladash 7dfd50d5cc cellSaveData: followup to #7652 2020-03-01 20:09:46 +03:00
Nekotekina b05b16aedc sys_ppu: Hotfix for detached threads 2020-03-01 18:30:16 +03:00
Eladash ffd5a9e91c cellSaveData: Add some error checks for fixedSet, fileSet params 2020-03-01 10:56:41 +02:00
AniLeo fbe6900b28 rpcs3_version: Bump to 0.0.9 2020-03-01 00:17:15 +03:00
kd-11 7fe9802f87 vk: Properly use declared pitch when loading simple images 2020-03-01 00:16:52 +03:00
kd-11 14aebeac58 video-out: Allow applications to successfully change display resolution
- Avoids a situation where a game configures output correctly but gets back bogus information later when querying.
- Should fix games being broken at some resolutions but not others.
2020-03-01 00:16:52 +03:00
kd-11 76bbbe27f1 vk: Fix dma resource leak
- Fix broken check; a relic of the past where flush method would reset the fence
2020-03-01 00:16:02 +03:00
kd-11 9af52d12a8 vk: Improve events
- Make events properly managed objects.
- Add a workaround for AMD's broken event status query
2020-03-01 00:16:02 +03:00
kd-11 5eb314fbbb vk: Add execution barriers.
- Useful for debugging
2020-03-01 00:16:02 +03:00
Eladash 655f7ce8a2 cellSaveData: Add null funcStat check
it's ordered specially for some functions
2020-03-01 00:14:45 +03:00
Eladash c11074a128 RawSPU: fix race condition in RunCntl stop request 2020-02-29 21:54:54 +03:00
InvoxiPlayGames ef6854ca46 sys_usbd: Guitar Hero Live controller emulation (#7336)
* Initial GHLtar emulation

* Add GHLtar to CMakeLists and VS project, zero the buffer and remove unused header values

* Fix coding style issues and include headers

* Remove redundant if, improve code formatting

* Remove needless includes

Co-authored-by: Ivan <Nekotekina@users.noreply.github.com>
2020-02-29 21:40:44 +03:00
clienthax 7e590eaa2f Change key_vault to use version ranges. 2020-02-29 21:24:15 +03:00
MSuih 94478ad4a0 Add error for missing firmware 2020-02-29 21:19:01 +03:00
Nekotekina e28e51463b Compilation fix
Should fix #7642
2020-02-29 21:16:28 +03:00
Nekotekina cb252b1ce2 Partial revert of 5871c4e93f 2020-02-29 18:39:15 +03:00
Nekotekina d37e770497 Shut up all channels on exit at some point
Some object are getting destroyed.
Makes valgrind more quiet.
2020-02-29 18:29:32 +03:00
Eladash 5871c4e93f Segfault/exceptions reports: Get rid of unhandled exeption handler, log memory bases
* Getting rid of handled exception handler fixes 2 things:
- Visual Studio debugger won't force it's own handler on unhandled exception.
- SPU segfaults in recompiler can now be reported.

* Log vm memory bases.
2020-02-29 17:16:36 +03:00
Eladash 50f51be06a Improve sys_timer_get_information (#7638)
* Improve sys_timer_get_information

* sys_timer_disconnect_event_queue sets STATE_STOP regardless of port connection status.
* sys_timer_get_information sets 0 for period and next_expire if the timer is stopped.

* Fix two minor races in lv2_timer thread

* If the timer thread is about to fire an event of queue x, then another thread disconnects the queue, then restarts the timer and connects the event queue, then the timer thread sends an event - event data combination (source, data1, data2, next) may be inaccurate.

* If the timer thread is about to send an event (periodically), then another thread stops the timer and starts it again with sys_timer_start_periodic_absolute, timer.expire in info->timer_state in sys_timer_get_information may be inaccurate.
2020-02-29 17:15:25 +03:00
Nekotekina 8e5a03f171 Use named_thread_group in rsx_cache.h 2020-02-29 16:55:25 +03:00
Nekotekina f72971f19f Implement named_thread_group 2020-02-29 16:55:25 +03:00
kd-11 eb140c52a4 rsx: Reset ZCULL statistics at the end of a frame
- Workaround for games that leak zpass/zstats.
  The information is useless anyway without a clear op so it should be fine.
2020-02-29 14:23:52 +03:00
kd-11 198c84cabf rsx: Fix zcull clear command; do not clear ZPASS when ZSTATS is cleared. 2020-02-29 14:23:52 +03:00
Eladash 5a73943be6 Fix sys_timer_destroy
Also some cleanup.
2020-02-29 13:06:14 +03:00
Eladash 34a0c3f488 cellSaveData: Add error param 72, 73 checks for file write ops 2020-02-29 13:06:14 +03:00
Eladash 8762f2a588 Use more starts_with 2020-02-29 13:06:14 +03:00
Eladash d7dd4897f8 Allow 0x30000000 > addr >= 0x2000000 ppu loader exec allocations (workaround) 2020-02-29 13:06:14 +03:00
Nekotekina a84077f174 sys_ss: use BCryptGenRandom on Win32 2020-02-29 12:15:42 +03:00
kd-11 08f3460365 vk: Fixup for RCB/RDB in special cases
- Images must be in TRANSFER_DST_OPTIMAL or GENERAL layouts to call the image upload routines.
2020-02-29 12:13:11 +03:00
Nekotekina be0e4e9879 Move Progress Dialog Server to named_thread
Make it a global variable, but that can be actually destructed at exit.
2020-02-28 21:59:56 +03:00
Nekotekina 799c3f9708 Remove global thread counter (again)
Seems fine without it now.
2020-02-28 21:50:19 +03:00
Nekotekina 490f58ff3c Try to purge thread_state::detached
It's rarely necessary, but can cause unexpected problems.
2020-02-28 21:11:13 +03:00
kd-11 cb047fcc75 rsx: Disable zstat checks to avoid unnecessary stream splitting (#7624) 2020-02-28 20:27:31 +03:00
Nekotekina ac2581659a RSXOffload: fix dma_manager::sync() freeze on exit
Its logic was completely broken.
2020-02-28 19:55:43 +03:00
Nekotekina f335d034fc Fix RSX Offloader thread exit (MTRSX fix)
Hangs on exit if MTRSX is enabled.
2020-02-28 19:43:42 +03:00
Eladash 05bb6e1545 Improve sceNpDrmVerifyUpgradeLicense(2), sceNpDrmIsAvailable(2)
* sceNpDrmVerifyUpgradeLicense(2): copied string from content_id is 47 chars in length at max, is no null term was found it forces string termiation.
* sceNpDrmIsAvailable(2): Same with drm path just that it's 256 chars at max.
2020-02-28 19:10:39 +03:00
Nekotekina 65eeee0f4c Remove cancerous lf_value<>
Replace thread names (generic, PPU, SPU) with new shared pointers.
Devirtualize cpu_thread::get_name (used in single case).
2020-02-28 18:54:46 +03:00
Nekotekina bf4bdf73b7 Remove unused lf_hashmap
There are better ways to implement this.
2020-02-28 18:54:46 +03:00
Eladash 30f7c81cc5 cellSaveData: Implement/Fix param error 22 for funcFile, funcDone, funcFixed and funcList 2020-02-28 16:47:51 +01:00
Nekotekina 08ab9c4b04 fixed_typemap.hpp: reset creation index to avoid confusion 2020-02-28 00:16:04 +03:00
Nekotekina 4474757162 fixed_typemap.hpp: remove <algorithm> dep in header
Create fixed_typemap.cpp
2020-02-28 00:04:37 +03:00
Nekotekina bacd1698fc fixed_typemap.hpp: minor cleanup 2020-02-27 23:04:23 +03:00
Ivan e133c7a7dc Merge pull request #7506 from elad335/patch-1
PPU exec/ovlm ldr: restrict allocations
2020-02-27 22:55:30 +03:00
Eladash 0eabfdcadd cellSavedata: reset padding of g_savedata_context 2020-02-27 22:31:31 +03:00
Eladash d86241bbcd cellSaveData: reset fileSet and fileGet->reserved everytime 2020-02-27 22:31:31 +03:00
Eladash 42a0512c66 cellSaveData: Avoid passing vm pointer to native API 2020-02-27 22:31:31 +03:00
Eladash 556aba46b5 cellSaveData: do not fail on empty directory 2020-02-27 22:31:31 +03:00
Stephen McKinney 2b853cc8bc Don't improperly resolve symlinks when booting games. 2020-02-27 22:30:11 +03:00
JohnHolmesII 00b74fb951 Update BUILDING.md and CMakeLists.txt
Several parts of the guide had fallen out of sync, notably the Ubuntu section. I've tried to clean it up a bit.
In addition, I matched some of the version numbers to what is found in the CI system here: https://github.com/hcorion/rpcs3-docker/blob/master/xenial/Dockerfile
2020-02-27 21:31:43 +03:00
Nekotekina ecd68dfc70 overlays: add "thread bits" to wait on and avoid lockup
Add TLS variable to store its own bit.
2020-02-27 19:14:08 +03:00
Nekotekina d0b2eecc9a fixed_typemap.hpp: polish move constructor a bit
GitHub, plz.
2020-02-27 19:14:08 +03:00
Nekotekina d5c85f308a fixed_typemap.hpp: forgot destructor 2020-02-27 13:04:12 +03:00
Nekotekina f71cdb4eb8 g_fxo: implement logging for object creation/destruction.
Only logged at automated phase for initial/final processing.
2020-02-27 13:04:12 +03:00
Megamouse ee46ad1ca9 move overlays code to headers 2020-02-26 23:43:18 +01:00
gamerforEA 762c106e19 sys_prx_.cpp: Fix RAII locks usage (without local variables they destroyed immediately after construction) 2020-02-27 00:38:55 +03:00
gamerforEA 93552a5958 Apply some Clang-Tidy fixes 2020-02-27 00:38:55 +03:00
gamerforEA c0fbf3091e Remove unnamed namespaces from headers 2020-02-27 00:38:55 +03:00
gamerforEA 49294a3dd2 Add missing include guards 2020-02-27 00:38:55 +03:00
RipleyTom abfa303e07 Auto-updater minifix 2020-02-26 22:06:58 +03:00
Nekotekina 5094ab8283 Fix RSX Offloader thread name 2020-02-26 21:57:01 +03:00
Nekotekina a0d2bc9769 named_thread: allow default constructor only with thread_name
In C++20, lambdas may be default-constructible allowing nasty stuff.
2020-02-26 21:23:30 +03:00
Nekotekina e102dc2a94 Add message on exit if some thread are still waiting 2020-02-26 21:23:30 +03:00
Nekotekina 5e59f18720 fixed_typemap: implement need<> method
It may be used in constructors of other objects to assert a dependency.
It also helps to ensure reverse destruction order of that dependency.
2020-02-26 21:23:30 +03:00
Nekotekina b35a5982e8 Fix one bug with MsgDialog thread (freeze on exit)
Forgot to check thread state
2020-02-26 21:23:30 +03:00
Eladash 4d3cdca7f6 Stub sys_spu_thread_group_set_cooperative_victims and syscall_253 2020-02-26 18:17:13 +00:00
Eladash 0d4f8ca234 fs: Make fs::get_dir_size able to report an error 2020-02-26 18:17:13 +00:00
kd-11 569e1c2df6 rsx: Fix typo. Noted by github user @gamerforEA 2020-02-26 19:40:35 +03:00
kd-11 6e9392fb45 rsx: Restructure ZCULL query triggers
- Both ZCULL stats and ZPASS stats require hardware queries, but
  ZCULL stats should not contribute to ZPASS stats and vice versa!

- Disables hardware queries for ZCULL stats by themselves, we cannot
  generate them correctly anyway and no game so far has been found to
  actually use them. Should lessen the load on the backend for games
  that do not actually require it.
2020-02-26 19:40:35 +03:00
Nekotekina df1813b4e2 overlays: hotfix for waiting on thread_count 2020-02-25 23:43:05 +03:00
Nekotekina ff16e678a5 Add thread_count instead of former thread pool 2020-02-25 23:16:55 +03:00
Nekotekina 982856e70d overlays: remove unused threadpool 2020-02-25 22:56:50 +03:00
Nekotekina 144c20649f Try to fix msg dialog breakage 2020-02-25 22:50:44 +03:00
Megamouse e719bcf338 overlays: add layer modes to osk 2020-02-25 21:57:49 +03:00
Megamouse 3f4226b70e overlays: Fix find and replace regression 2020-02-25 21:57:49 +03:00
Megamouse 620cfd5063 overlays: move code to overlay_utils.cpp 2020-02-25 21:57:49 +03:00
Megamouse 69b2e746fa overlays: fix overlay_utils.h filter 2020-02-25 21:57:49 +03:00
Megamouse 2341749485 overlays: add overlay_osk.h 2020-02-25 21:57:49 +03:00
MSuih 33abcf74f2 Add information about boot process 2020-02-25 19:42:20 +03:00
Nekotekina 4e33ae319b fixed_typemap.hpp: minor fixup 2020-02-25 18:27:05 +03:00
Nekotekina b083edccd3 Threads: remove some unused code (remnants from thread spawn) 2020-02-25 15:57:57 +03:00
Nekotekina b59bb16fad Threads: removed outdated on_abort detection deemed unnecessary
May cause regressions.
2020-02-25 15:48:25 +03:00
Nekotekina 3f943945a7 Threads: Remove unused on_wait() detection to simplify code 2020-02-25 15:36:08 +03:00
Nekotekina 136d769895 Fix g_fxo->init internal logic for CTAD (use () not {})
Also improve comments to the functions.
2020-02-25 14:34:06 +03:00
Nekotekina 9c9c2eb2c9 Fix wrong g_fxo->init_crtp name, use just init<> 2020-02-25 14:07:50 +03:00
Nekotekina 318a364d09 Try to fix OSK 2020-02-25 14:03:13 +03:00
Nekotekina fa02a04baa Add g_fxo->init_crtp to simplify thread construction 2020-02-25 11:51:41 +03:00
Nekotekina 7eebe06931 Restore thread counter (world may be not ready yet)
Remove dumb 1300ms timeout.
2020-02-25 11:51:19 +03:00
kd-11 cd40bc8c61 overlays: Avoid race condition between rendering and layout operations for system widgets
- System widgets are callable from outside RSX code.
- Responding to draw requests while setup is in progress can cause malformed cached output
- Fixes glitched layouts for system message dialogs
2020-02-24 23:33:47 +03:00
kd-11 f6ebd88687 overlays: Ditch wstring for u32string
- Turns out wstring is not the same as u32string on windows.
2020-02-24 23:33:47 +03:00
MSuih 13ef0cc8c4 Fix native UI tooltip 2020-02-24 19:45:16 +01:00
MSuih df5059b142 Change logging level for invalid disc path 2020-02-24 19:45:16 +01:00
Megamouse f7666f44da Untangle GUI and input includes 2020-02-24 16:31:01 +01:00
Megamouse 7b49249f5f Input: Add config lerp factor for buttons and triggers
Adds new lerp factors to the keyboard pad handler In order to simulate triggers and analog buttons.
See "Analog Button Lerp Factor" and "Trigger Lerp Factor" in the yml in InputConfigs/Keyboard/.
Values Range from 0-100 as before, where 100 is instant press and 0 is never.

Currently I'm not planning any GUI element for this.
2020-02-24 08:56:57 +01:00
Megamouse 64ed2f1151 Input: use std::lerp instead of lerp template 2020-02-24 08:56:57 +01:00
Megamouse 17f335648c Input: misc updates to some functions in PadHandler 2020-02-24 08:56:57 +01:00
Nekotekina 1dc2eb1cc8 Avoid deprecated av_register_all with version check. 2020-02-23 20:58:02 +03:00
Emmanuel Gil Peyrot 1a702de9e4 cellVdec: replace deprecated ffmpeg function
avcodec_decode_video2() is deprecated, now replaced with
avcodec_send_packet() and avcodec_receive_frame().
2020-02-23 20:21:59 +03:00
Nekotekina f1241c572c Fix some deprecation warnings (plunder cellAdec btw) 2020-02-23 20:21:59 +03:00
Eladash 522daf5eac rsx: Fix NULL renderer 2020-02-23 19:57:55 +03:00
Nekotekina e772dde3cc Add more thread information (context switch, page faults).
Only implemented on Linux, possibly works on some BSD.
2020-02-23 16:21:26 +03:00
Nekotekina 8b4b859091 Remove "thread_ctrl::spawn" 2020-02-23 15:03:38 +03:00
Nekotekina 18db020b93 Fix warning in RSXOffload.cpp (rewrite thread) 2020-02-23 14:19:23 +03:00
Nekotekina 7069e7265f RSX: move g_dma_manager to g_fxo 2020-02-23 13:12:50 +03:00
Nekotekina fa0bf6a92c Fix "unknown pragma" on zlib clang workarounds 2020-02-23 10:42:35 +03:00
JohnHolmesII 758902382d AudioBackend.cpp: Implicit enum to float conversions are deprecated 2020-02-23 09:38:04 +03:00
JohnHolmesII cc71d2c4bf memory_string_searcher: Change to std strings
- Compiler warnings indicated that the call to toStdString() did not
   create an object that lived long enough be used. Simply use std
   string ahead of time.
2020-02-23 09:38:04 +03:00
JohnHolmesII 9b7d28b5dd cellSaveData.cpp: Use ref instead of copy 2020-02-23 09:38:04 +03:00
JohnHolmesII b9ee53d82a game_list_frame.cpp: Fix progress dialog constructor arguments 2020-02-23 09:38:04 +03:00
JohnHolmesII 479a64c4e8 Remove some inline compiler pragmas 2020-02-23 09:38:04 +03:00
JohnHolmesII 7b54d386f2 Set required Clang to 9
- 10 is not yet required and is not very available
2020-02-23 09:38:04 +03:00
JohnHolmesII 7eccbecb2f File.cpp: Make var ref instead of copy 2020-02-23 01:22:38 +01:00
Nekotekina 2bab3afae0 Fix deprecation warning in System.cpp 2020-02-22 19:41:03 +03:00
AniLeo fa3fde7a29 hle: Fix cellAvconfExt function registrations
Co-authored-by: Clienthax <clienthax@gmail.com>
2020-02-22 17:27:58 +03:00
AniLeo 3aa293a7a3 hle: cellAuthDialog
Basic RE of cellAuthDialog, stubs functions
2020-02-22 16:03:14 +03:00
RipleyTom 4befa36365 Use official libusb repo 2020-02-22 16:02:40 +03:00
Nekotekina 96be40bf30 Implement cellMsgDialog closing thread
Fixing deprecation warning.
2020-02-22 15:17:02 +03:00
Nekotekina 3001701059 Test: disable warning for "macro redefined" spam 2020-02-22 15:17:02 +03:00
kd-11 fa41297b27 overlays/trophy: Migrate to multibyte strings 2020-02-22 15:07:14 +03:00
kd-11 b8f51398b7 overlays/save_dialog: Migrate to multibyte strings 2020-02-22 15:07:14 +03:00
kd-11 cb2129c7e4 overlays/osk: Migrate to multibyte encoding 2020-02-22 15:07:14 +03:00
kd-11 703ec9f896 overlays: More unicode utilities 2020-02-22 15:07:14 +03:00
kd-11 19350d024b overlays: Font system improvements
- Add support for Hangul blocks (korean)
- Restructure font fallback system to allow the user to 'install' fonts if missing.
  Should allow fonts to work with no firmware on open systems like linux
2020-02-22 15:07:14 +03:00
kd-11 8e68427daf overlays: Add basic font substitution system and separate JPN from Latin-1 set
- Gets JP glyphs to render correctly, but the generalization may negatively affect other CJK glyph sets.
  PS3 doesn't seem to use other glyph sets much however.
2020-02-22 15:07:14 +03:00
kd-11 6220206cbc vk: Implement 2D array textures required for new font subsystem 2020-02-22 15:07:14 +03:00
kd-11 1df1ceb4ea gl: Support new glyph format with array textures 2020-02-22 15:07:14 +03:00
kd-11 6178a0ab25 overlays: Migrate to wide-char strings 2020-02-22 15:07:14 +03:00
Eladash 6bb083a77c Add more information for segfault reports (#7538) 2020-02-22 10:58:42 +00:00
Megamouse 171e6c6e54 Qt: remove duplicate paths in game list 2020-02-21 21:00:46 +01:00
Megamouse 313b967217 Qt: simplify blockingMap occurances 2020-02-21 21:00:46 +01:00
Megamouse b47a8b9995 Skip some logging in add_only Load 2020-02-21 21:00:46 +01:00
Megamouse c4a1c6f845 Do not reset PS3_GAME when booting disc patches 2020-02-21 21:00:46 +01:00
Eladash 24d3e2b2b8 cellGameDataCheckCreate(2): More improvements
* IsNewData is false only if PARAM.SFO exists.
* Don't create directory on error (setParam == nullptr && isNewData).
* Return CELL_GAMEDATA_ERROR_BROKEN if PARAM.SFO exists on target directory yet is not from GD category.
2020-02-21 21:15:55 +03:00
Nekotekina 145b80d262 Re-enable warning "macro redefined" (clang) 2020-02-21 19:36:05 +03:00
Nekotekina 5e75a0c497 Disable cotire on travis
Make some workarounds for clang because it poorly supports -Wold-style-cast
2020-02-21 17:03:54 +03:00
Nekotekina 972e0ab31d Remove -Wno-reorder and make it an error 2020-02-21 15:20:34 +03:00
Eladash 54f2c27ba0 cellGameDataCheckCreate(2): Set nullptr in setParam
* setParam is nullptr by default.
* if setParam is null and it's new data, return error code.
2020-02-21 14:49:45 +03:00
Eladash 0ba1f8f4ef Fix cellDiscGameGetBootDiscInfo on error
* Always set first dword to 0, other bytes are untouched,
2020-02-21 14:49:45 +03:00
Eladash 4d7e53d7a0 cellGameDataCheckCreate(2): Check dirName 2020-02-21 14:49:45 +03:00
Eladash 0c757222cb Minor improvements to sys_ss_random_number_generator
* Replace CELL_EABORT with exception.
* Improve error codes a bit.
* pkg_id == 1 should not return an error if root permission is present.
* Avoid passing vm pointers to native API, use temp buffer instead.
2020-02-21 14:49:45 +03:00
Megamouse 0ed87be47a Qt: add apply button to settings dialog 2020-02-21 10:08:22 +01:00
Silent e005581dda Gracefully fall back to a null renderer if XAudio2 fails to Init
This can happen as a non-error condition if user has no output
audio devices enabled.
2020-02-20 23:33:09 +03:00
Silent 312fc94daa Replace XAudio2 implementation with an unified Xaudio2Redist
This removes dual implementation for 2.7 and 2.8/2.9 interfaces
and also removes reliance on DirectX End User Runtimes for Windows 7.
2020-02-20 23:33:09 +03:00
Megamouse 0dd417e5f2 Add more game window title options 2020-02-20 20:07:25 +01:00
Eladash dd85e733d3 Fixup for #7304 2020-02-20 20:31:56 +03:00
Nekotekina 9e9ddf42c9 Change stdafx.h to trigger pch regeneration 2020-02-20 17:42:28 +03:00
Nekotekina 4d1f818162 umax: restore "natural" operation order 2020-02-20 17:13:41 +03:00
Nekotekina 3e0e1f668c Another attempt to fix OSX 2020-02-20 16:54:56 +03:00
scribam 4662ee0255 cmake: Update glslang build options 2020-02-20 16:23:25 +03:00
Nekotekina 2f255a528e Another attempt on umax 2020-02-20 15:18:31 +03:00
Nekotekina 987223da3f Minor fixup for /dev_flash creation 2020-02-20 14:38:09 +03:00
Nekotekina 78cc36cdd8 Allow empty /dev_flash cfg (fixup) 2020-02-20 14:07:26 +03:00
Nekotekina 580b5cda0e Revert "Maintenance: disable cotire on travis (Linux)"
This reverts commit 48c47eede4.
2020-02-20 13:45:44 +03:00
Nekotekina 81f974125c Travis: allow failure for osx 2020-02-20 13:13:48 +03:00
Nekotekina 48c47eede4 Maintenance: disable cotire on travis (Linux) 2020-02-20 13:02:57 +03:00
Nekotekina da036de3e4 Restore some /dev_flash logic 2020-02-20 12:43:53 +03:00
Nekotekina 92e3eaf3ff Fix signed-unsigned comparisons and mark warning as error (part 2). 2020-02-19 22:54:58 +03:00
Nekotekina 771eff273b First part of fixing sign-compare warning (inside be_t). 2020-02-19 22:54:58 +03:00
Nekotekina 0cf35e3b22 Implement umax global variable (max unsigned value)
Implements operators == and != comparisons.
2020-02-19 22:54:58 +03:00
Nekotekina 70bcb1cd52 Add "-Wno-macro-redefined" for clang spam 2020-02-19 21:17:23 +03:00
Nekotekina 762c8d5430 Remove -Wno-sign-compare 2020-02-19 21:17:23 +03:00
Zion Nimchuk aa9055f4c3 Switch the AppImage building over to gcc from clang
Turns out the current version of clang doesn't support the [[likely]] and [[unlikely]] attributes, so to ensure good performance, we'll be switching to gcc, at least for now.
2020-02-19 21:16:32 +03:00
Zion Nimchuk 9d1833c5a8 Bump FAudio depedency, set FAudio to build statically, enable FAudio in the build script 2020-02-19 21:16:32 +03:00
AniLeo 583220b95a .gitignore: maintenance, add missing files 2020-02-19 21:15:52 +03:00
AniLeo 85bde0f43f themes/YoRHa: Workaround broken Trophy Manager bg 2020-02-19 21:15:52 +03:00
AniLeo b96f064868 OpenAL: Update to 1.20.1 2020-02-19 21:15:12 +03:00
Eladash 6de91a1691 HLE cellGcmSys: Make IOTable accurate
Affects cellGcmAddressToOffset when using addresses above 0xC0000000
2020-02-19 18:11:30 +00:00
Eladash 1aa11440e0 HLE cellGcmSys: Make cellGcmUnmapEaIoAddress accurate 2020-02-19 18:11:30 +00:00
Eladash df8d0cde4a RSX/SPU: Accurate reservation access 2020-02-19 18:11:30 +00:00
Zion Nimchuk 57a9844279 Re-eanble gcc matrix in Travis CI, thanks to Neko for the hint 2020-02-19 10:08:58 +03:00
Eladash f02b4801b2 Fix max SPURS threads regression 2020-02-18 19:20:40 +00:00
Eladash 727d783959 RawSPU: protect NPC from writes/reads in running state 2020-02-18 18:09:10 +00:00
Eladash fad8b38b28 sys_spu: protect sys_spu_image members in kernel mode
Save relevant info in idm, set sys_spu_image segs and nsegs members to 0.
2020-02-18 18:09:10 +00:00
Nekotekina 8a176de6a1 Restore -Wenum-compare and fix some [=] warnings 2020-02-18 17:37:30 +03:00
Nekotekina 0ee2f761ae Fix warning in lf_fifo<>::push_begin() 2020-02-18 14:59:11 +03:00
Nekotekina c48ceafc15 sys_sync.h: fix warning (signed prio) 2020-02-18 14:53:23 +03:00
Nekotekina ee6494c14b Use strcpy_trync in cellAvConfExt.cpp (silence warnings) 2020-02-18 14:53:23 +03:00
Nekotekina a1456da24e Silence C++17 std::iterator deprecation warning 2020-02-18 14:53:23 +03:00
Nekotekina f08c778d2c Use more starts_with/ends_with.
Remove ends_with global func.
2020-02-18 14:53:23 +03:00
Megamouse 90f4023cb8 appv 2020-02-18 09:14:14 +00:00
Silent ea9abe7701 Qt: Update Game List Icon on changing Game List Mode 2020-02-18 00:25:21 +01:00
Nekotekina 5c42d29c98 Try to fix MSVC warning (std::iterator deprecation) 2020-02-17 22:01:11 +03:00
Nekotekina 950940febe cheat_manager: minor fix for T to be_t transition 2020-02-17 22:00:32 +03:00
Nekotekina 6e7fbc5c5c endian.hpp: fix zero array warning 2020-02-17 22:00:00 +03:00
Nekotekina 6a1a0bf48d Use std::endian for endianness test
Remove legacy IS_LE_MACHINE IS_BE_MACHINE macro.
2020-02-17 21:33:24 +03:00
Nekotekina 244e74ebe2 Try to ignore some annoying warning (seems CIB) 2020-02-17 20:56:03 +03:00
Silent aa14432846 Disable vertex cache checkbox with MTRSX 2020-02-17 20:34:07 +03:00
Ani afb594c233 gitmodules: Fix LLVM branch
gitmodules contained some old unused branch
2020-02-17 20:29:44 +03:00
Megamouse 7a7ac625cd move enum formatters from system to config files 2020-02-17 15:08:17 +03:00
Megamouse fe75311be2 move config structs to own files and clean up some headers 2020-02-17 15:08:17 +03:00
Eladash 812d03894b PPU exec/ovlm ldr: restrict allocations 2020-02-16 22:48:23 +02:00
kd-11 5e6b1003ec vk: Only declare explicit subpass dependencies for RADV 2020-02-16 18:00:06 +03:00
clienthax a08ed0d22e Fix #7445 2020-02-16 17:09:32 +03:00
Megamouse b5ed73ebe0 Qt: add reset button to game window title and center the label 2020-02-16 13:56:49 +01:00
Eladash c1bdaccd8c sceNpTrophyRegisterContext: Fix off by one progress callbacks count 2020-02-15 23:32:29 +01:00
Eladash 4421831c8b sceNpTrophyRegisterContext: Fix values passed to first callback 2020-02-15 23:32:29 +01:00
Eladash d03804b523 Fix sceNpTrophyGetTrophyInfo
* Only writeback data on success.
* Fix a typo on error code of invalid trophy ID.
2020-02-15 23:32:29 +01:00
Nekotekina 634c4355fe Fix startup failure on invalid games.yml
Add some exception checking/ignoring.
2020-02-15 22:34:10 +03:00
Megamouse ee54ba970a GUI: add custom title format to settings dialog 2020-02-15 20:33:02 +01:00
kd-11 23f1515448 vk: Explicitly declare null subpass dependencies
- We do not want any actual dependencies, but it turns out removing them
  entirely makes the driver add even worse dependencies.
2020-02-15 21:45:25 +03:00
Nekotekina 4018b833ad game_list: fix duplicate removal from games.yml
Also add some warnings.
2020-02-15 14:08:08 +03:00
Eladash ddf87864de atomic_t: Fix regression from #7489 2020-02-15 14:07:52 +03:00
Eladash 299af768e8 HLE cellGcmSys: Make cellGcmAddressToOffset accurate 2020-02-15 14:07:52 +03:00
Megamouse e645627b78 Qt: Allow for duplications in game list
This fixes app versions when multiple game data directories were found.
We only removed duplications because we didn't wanna see multiple disc games from different locations
2020-02-15 11:36:01 +01:00
Megamouse 1cb1d14d0c Qt: only add version update hint to bootable applications 2020-02-15 11:36:01 +01:00
Megamouse 1c2df15755 Qt: simplify category localization in gamelist refresh 2020-02-15 11:36:01 +01:00
Megamouse 687bb1697b Qt: fix gamelist version check regression after localization changes 2020-02-15 11:36:01 +01:00
Eladash 04e0bf2eff Whitespace fix after #7087
Was this close to enter programmers' hell.
2020-02-15 11:37:13 +03:00
Eladash eb8710d3c1 atomic.hpp: C-style casts cleanup 2020-02-15 11:37:13 +03:00
Eladash e98fcfdf77 rsx debugger: Fix a crash on opening before rsx was intialized 2020-02-15 10:41:42 +03:00
Eladash cdda19c79f Fix recursive locking in sceNpTrophyUnlockTrophy 2020-02-15 10:41:15 +03:00
Eladash fa9330d0e0 Log returned reqspace in sceNpTrophyGetRequiredDiskSpace 2020-02-15 10:41:15 +03:00
Eladash 1d4595a349 Idm: Minor assert fix 2020-02-15 10:41:15 +03:00
Eladash 9344b21484 rsx: Unify FIFO recovery methods
TODO: Maybe consider fifo stack content when recovering.
2020-02-14 17:11:26 +03:00
Eladash 07f300a14e rsx: ZCULL typo fix 2020-02-14 17:11:26 +03:00
Eladash ddeb39d8de HLE cellGcmSys: Fix unmapping 2020-02-14 17:11:26 +03:00
Nekotekina 0d7aa5e310 GUI: implement custom title format
New option "Window Title Format" in Misc.
Backward compatible with FPS disabler.
Make rpcs3:::get_branch() return string_view.
2020-02-13 21:24:52 +03:00
Eladash 606693a9f7 Avoid closing the emulator after access violation 2020-02-13 14:14:28 +03:00
Eladash 78c49e7331 cellSearch: another memory access fix 2020-02-12 20:02:18 +03:00
Eladash 9760053c8c cellSearch: Fix id memory access (#7476) 2020-02-12 18:17:45 +03:00
Nekotekina e8988faed5 geometry.h: remove MSVC workaround 2020-02-12 12:50:42 +03:00
Nekotekina 7137142351 geometry.h: more cleanup
Remove wrong constructors.
2020-02-12 12:50:42 +03:00
Zion Nimchuk bacd234258 Add a check for GLIBCXX_3.4.26 support in the AppImage checker binary 2020-02-12 12:49:46 +03:00
Silent 3006b003c4 Implement links as a cellSearch specific concept
Linking in VFS is done only from cellSearchPrepareFile and works
by mounting virtual files to host FS files
2020-02-12 12:49:02 +03:00
Silent e30637351e Move SearchState to a fxo object so it resets with emulation 2020-02-12 12:49:02 +03:00
Silent d2b83c69bb cellSearch updates from Brolijah
Co-authored-by: Brolijah <brolijahrh@gmail.com>
2020-02-12 12:49:02 +03:00
Nekotekina bcbe324534 geometry.h: make conversion operators explicit
It requires static_cast<> to call them.
2020-02-11 13:21:45 +03:00
Eladash dcb30df7c8 rsx capture: Fix capture recovery after a crash 2020-02-10 21:39:39 +00:00
Eladash bdab26ec09 rsx: rewrite io mappings
Along with some with fixes to cellGcmSys HLE.
2020-02-10 21:39:39 +00:00
kd-11 f47333997f rsx: Validate memory blocks before checking for overlap 2020-02-10 21:48:35 +03:00
kd-11 3787108ee7 rsx: Typo fix in audit condition 2020-02-10 21:48:35 +03:00
Megamouse 30d176ac5e Qt/linux: set DISPLAY variable if undefined 2020-02-10 21:48:13 +03:00
RipleyTom 98f91457bf Small sys_usbd changes 2020-02-10 21:47:48 +03:00
Megamouse 6847e52364 Fix visual studio filters after someone tinkered with the files 2020-02-10 21:47:13 +03:00
Zion Nimchuk 896d16ec7b Bump minimum Qt5 version to 5.14.0 in CMake 2020-02-10 21:46:35 +03:00
Megamouse 6862790cf7 Qt: icon overhaul 2020-02-10 17:38:19 +01:00
Eladash 639245c071 Make handle_access_violation noexcept 2020-02-10 17:27:34 +03:00
Nekotekina 034267adb2 Compilation fix 2020-02-10 16:57:56 +03:00
Nekotekina 6e47a6fb76 Update LLVM for C++2a workaround 2020-02-10 14:47:12 +03:00
Nekotekina 9569ae24e0 Bump minimal compiler versions: gcc-9 and clang-10. 2020-02-10 14:47:12 +03:00
Nekotekina 491526b421 Add option USE_COTIRE=ON (by default)
Precompiled headers cause rebuild problems with ninja, for example.
2020-02-10 14:47:12 +03:00
Nekotekina 5a41d75eb8 Silence unused parameter warning 2020-02-10 14:47:12 +03:00
Nekotekina 4bc431ec31 Silence deprecation warning (implicit capture of this on [=]) 2020-02-10 14:47:12 +03:00
Nekotekina 1bc9fd2863 Set cmake min version and CXX_STANDARD to 20 2020-02-10 14:08:51 +03:00
Megamouse 5d82b0f4c4 Qt: set min version to 5.14 2020-02-10 14:05:36 +03:00
Zion Nimchuk d3abff5486 Disable FAudio due to upstream packaging issues 2020-02-10 13:31:29 +03:00
Zion Nimchuk 77aa1fbeea Bump glslang to fix issues with LLVM 10 2020-02-10 13:31:29 +03:00
Zion Nimchuk af9bc631cc Bump Docker version, update clang10+gcc9, CMake 3.16, adds SDL2, LLD
The vulkan library and SDL2 libraries are now self-built, due to distro packages being too low version
2020-02-10 13:31:29 +03:00
RipleyTom 762718002e make decrypt default to All Binaries 2020-02-10 09:45:06 +01:00
Maxim Kulyk 16bbd93885 [props] Move stdcpplatest and FH4 to common_default.props
This way stdcpplatest  and FH4 disabling propagates to all projects except llvm and glslang.
2020-02-09 17:32:02 +03:00
Eladash 80eff58950 cellAudio: Implement cellAudioSet/RemoveNotifyEventQueueEx 2020-02-09 12:31:55 +00:00
Nekotekina 7ea4eb0095 Atomic fix
Fix possible pointer arithmetic ops.
Fix fat atomics (currently unused).
2020-02-09 14:09:29 +03:00
kd-11 efc8c3f4a9 vk: Fixup for VK_ERROR_SUBOPTIMAL_KHR
- break from a switch does not break out of the external scope!
2020-02-09 13:45:30 +03:00
kd-11 792c481f6d rsx/overlays: Fix clipped rendering of UI elements
- Take viewport offset into account when applying window transforms.
  This is necessary because gl_FragCoord is based on the framebuffer and not the viewport.
2020-02-09 12:55:56 +03:00
13xforever 7973b01b1c update issue templates 2020-02-09 08:29:33 +00:00
Eladash 11675d7645 Compilation fix for VSH pr 2020-02-09 06:48:16 +00:00
Eladash 1915fe75a4 VSH: Stubs 2020-02-08 23:07:03 +03:00
Eladash 9d1bb60ad7 cellGcm HLE: fix cellGcmMapMainMemory
Fix arguments order, softcode RsxReports::report offset.
2020-02-08 22:18:56 +03:00
Eladash b7043ce000 Make rsx::get_address report caller location 2020-02-08 22:18:56 +03:00
kd-11 c64935f9dd rsx: Clean up graphics state notifications and add notification for change in point size
- Adds a backend notification when point size changes.
- Refactors all those separate notifiers into one reusable template.
2020-02-08 18:13:05 +03:00
Eladash 629eddfb9f sceNpTrophy: Implement SCE_NP_TROPHY_ERROR_CONTEXT_NOT_REGISTERED 2020-02-08 11:11:59 +00:00
Eladash 1f94c8f272 sceNpTrophyGetGameProgress Fix 2020-02-08 11:11:59 +00:00
Megamouse 0c8611bd49 Qt: fix game category localization 2020-02-08 11:04:13 +01:00
Megamouse 5dcb91b671 Restart games with the same config instead of global 2020-02-08 11:04:13 +01:00
Megamouse 7abda27b46 Fix Boot inconsistencies for Reloads 2020-02-08 11:04:13 +01:00
Megamouse 901fc87bca Only start the playtime clock if it makes sense 2020-02-08 07:13:29 +01:00
kd-11 54da9ac7e5 overlays: Fixup
- Avoid calling join on self thread.
- Avoid use-after-free.
2020-02-07 19:28:41 +03:00
kd-11 e45360de2b overlays: Fix use after free
- Overlay can be closed when secondary thread is asleep!
  Wait for it to wake before proceeding with deletion.
2020-02-07 16:15:02 +03:00
kd-11 d59c449ff6 vk: Remove an overzealous assert 2020-02-07 16:15:02 +03:00
MSuih 17df6c8878 Enable C++20 for MSVC in CMakeLists.txt 2020-02-06 22:15:58 +00:00
eladash f901846acb RawSPU: execute MFC proxy cmd after reading CMDStatus
Implement MFC proxy argument sequence checking.
2020-02-06 20:43:38 +00:00
Megamouse edcd2fc14a Qt: fix game grid regression 2020-02-06 19:58:19 +01:00
Megamouse 382bdcdcb7 Qt: set Tooltips.h to UTF-8 in order to fix translation with special characters 2020-02-06 17:41:50 +01:00
Megamouse 1bbc60c3e7 Qt: do not use localized filenames for default current config and default stylesheet 2020-02-06 17:41:50 +01:00
Megamouse c13d345604 Qt: add language menu 2020-02-06 17:41:50 +01:00
Megamouse efe907ffae Qt: use config to load translation file on startup 2020-02-06 17:41:50 +01:00
kd-11 0bba04ef8d vk: Fix a bug in RCB/RDB when MSAA is set to disabled.
- Initially MSAA option was hardcoded to be always enabled, this bug is a remnant of that time.
2020-02-06 17:54:05 +03:00
kd-11 43dae6c14d gl: Implement RCB/RDB 2020-02-06 17:54:05 +03:00
kd-11 2b5c24b304 gl: Fix memory barrier implementation and stub for RCB/RDB
- It's a miracle it even compiled
2020-02-06 17:54:05 +03:00
kd-11 50b1e26b17 gl: Fix a long-standing regression with typeless transfer caused by a typo.
- The parameters for the final upload should be 'unpack_info' not 'pack_info'!
2020-02-06 12:44:46 +03:00
kd-11 18e0559438 gl: Fix per-level sub-image sizes to comply with OpenGL guidelines for compressed textures 2020-02-06 12:44:46 +03:00
Eladash 37513b1898 SPU reservations: Do not access violate under vm::writer_lock
TODO: Throw exception when encountering page faults notification enabled memory
2020-02-06 00:27:17 +00:00
Silent fbbad7c851 Include trailing separators in section split
This fixes the case on Windows where one of the paths ends up consisting
only of a drive letter and no trailing slash - in which case Windows
treats it as "current directory on drive X" and not "root of drive X"
and GetFileAttributes throws an invalid param error
2020-02-05 23:21:32 +00:00
Eladash f8b3c48af7 sys_spu: Implement proper SPU group flags (#7320)
* sys_spu: Implement proper SPU group flags
2020-02-05 20:46:05 +00:00
kd-11 3cc42c1bf8 gl: Fix broken image transfer operations 2020-02-05 18:18:09 +03:00
kd-11 b6422c9a33 rsx: Fixup
- Destination Y coordinate must be 'rebased' onto the current slice by subtracting its offset.
  Only the local path was affected this time
2020-02-05 18:18:09 +03:00
Eladash 049e392a97 Make preferred spu threads dynamically adjustable 2020-02-05 10:06:07 +00:00
Eladash 9a64d08c9f Make sleep timers accuracy dynamically adjustable 2020-02-05 10:06:07 +00:00
Nekotekina c0f80cfe7a Use attributes for LIKELY/UNLIKELY
Remove LIKELY/UNLIKELY macro.
2020-02-05 10:42:34 +03:00
Eladash 49e11b7cfd cellVdecQueryAttrEx: Add some error checks for MPEG2 2020-02-05 05:01:07 +00:00
Eladash 6a32ceaab5 cellVdecQueryAttrEx: Add workaround for codec specific info 2020-02-05 05:01:07 +00:00
Eladash acc7320cae Fix cellVdecGetPicItem
Fix potential overflow, race condition and correctness fixes for picInfo_addr
2020-02-05 05:01:07 +00:00
Nekotekina 1a78e0e80c Make RPCS3 compile in C++2a mode 2020-02-04 23:43:55 +03:00
Eladash e9e8f0c5b7 cellGame: report not found sfo params 2020-02-04 18:29:52 +03:00
Eladash cb52ee0a4d cellGame: report fs::remove_all failure 2020-02-04 18:29:52 +03:00
Eladash 4488312e81 Avoid out of memory with cellGameGetParamString 2020-02-04 18:29:52 +03:00
kd-11 9d9b5c4d66 rsx: Rewrite coverage test to take sum of areas into account.
- TODO: A proper sweep algorithm to calculate sum of overlapping rectangles
2020-02-04 16:20:52 +03:00
kd-11 b9ec012922 rsx: Allow for proper data checks when WCB/WDB is enabled 2020-02-04 16:20:52 +03:00
Megamouse d47a8b49a4 Qt: use current locale for last played in gamelist
This also sets the basic groundwork for Qt translations
2020-02-04 09:18:05 +01:00
Megamouse 1759d6d90a Qt: fix gamelist sorting for playtimes 2020-02-03 21:22:11 +01:00
Nekotekina c4a01875d0 Space fix commit 2020-02-03 11:16:26 +03:00
Nekotekina f9a8efe406 SPU LLVM: gisable NewGVN pass
It goes into an endless loop with memory leak for some reason.
2020-02-03 11:16:03 +03:00
Silent 7f4e546f19 Protect m_storage.find(key) to fix a race 2020-02-02 22:28:14 +03:00
Nekotekina 0a2874405d logs: allow disabling RPCS3.log.gz
Disabled by creating a directory with the same name.
2020-02-02 14:32:29 +03:00
Nekotekina 87a5dd66ab Move logs::channel registration out of the constructor
Allow constinit initialization of logs::channel.
2020-02-02 14:12:54 +03:00
Eladash e57c01907e cellVdec: Improve cellVdecQuery and cellVdecOpen 2020-02-02 09:01:32 +03:00
Nekotekina 32df809626 Update llvm (fixup) 2020-02-01 19:05:36 +03:00
kd-11 7d2ed9200d rsx: Remove sections that are wholly inherited by new blocks
- Allows sections reclaimed by the surface store due to overlap/inheritance to be identified and removed.
- Additionally, potentially lowers the number of flushes required per block with multiple overlaps improving efficiency and theoretically performance.
2020-02-01 15:14:29 +03:00
Nekotekina f6e90b4c72 Fix FAudio logging 2020-02-01 13:34:36 +03:00
InvoxiPlayGames c1180d76dd sys_usbd: Fix bug preventing multiple USB devices 2020-02-01 12:34:42 +03:00
Nekotekina 6dfd97f0b6 Modernize SPU logging (spu_log variable) and remove log legacy
Remove legacy macro (LOG_ERROR, etc)
2020-02-01 11:52:52 +03:00
Nekotekina 327bb2d8f0 Modernize PPU logging (ppu_log variable) 2020-02-01 11:52:24 +03:00
Nekotekina 21f7b0ff0f Remove HLE log channel 2020-02-01 11:52:24 +03:00
Nekotekina 15391f45d0 Modernize RSX logging (rsx_log variable) 2020-02-01 11:52:22 +03:00
Nekotekina 3c0bd821c8 Give log channels fancier names
Improve LOG_CHANNEL macro to accept custom name.
2020-02-01 10:43:43 +03:00
Nekotekina ec80932c21 logs: use relaxed atomics
May help with optimizations.
2020-02-01 10:30:03 +03:00
Nekotekina 3eca2d5d6c Remove legacy LOADER log channel 2020-02-01 07:49:38 +03:00
Nekotekina d9a0619ddd Remove legacy GENERAL log channel
Add some more log channels instead.
2020-02-01 07:49:38 +03:00
Nekotekina efafda2650 Add config to silence all logs 2020-02-01 07:49:38 +03:00
Eladash 943368912b Hotfix after #7351 2020-02-01 05:50:58 +03:00
Nekotekina 1d0f359406 logs: add more log channels instead of GENERAL 2020-01-31 16:44:48 +03:00
Nekotekina d5f019c3d3 Implement logs::silence
Disables all log channels.
Also disables unsupported "default" log level for log channels.
2020-01-31 16:44:48 +03:00
Nekotekina 67075dfc6c logs: cleanup for audio backends
In process of removing GENERAL log channel.
2020-01-31 16:44:48 +03:00
Nekotekina a867522b16 logs: implement logs::get_channels() 2020-01-31 16:44:48 +03:00
Nekotekina 26cccead6e logs: remove legacy MEMORY channel
Add channels vm_log, sig_log.
2020-01-31 16:44:48 +03:00
Asinine e6f7467f67 Update missing rap file error 2020-01-31 14:13:55 +01:00
kd-11 36d5db7f30 rsx: Plug texture data leak in the 'exact match' path.
- Followup to previous texture data leak fix for the replaced section path.
2020-01-31 14:56:53 +03:00
Nekotekina e7b24461ec Implement logs::get_level 2020-01-31 12:09:52 +03:00
Nekotekina 007a7a5859 Fixup for LOG system.
Register all channels at program initialization and allow duplicates.
2020-01-31 12:09:52 +03:00
Nekotekina 59a0f810b9 Implement fat atomics
Atomics with embedded mutex bit.
2020-01-31 12:09:52 +03:00
Silent 9f678cc47a Fix code relying on initialization order
Allows Debug - LLVM to boot
2020-01-31 11:23:55 +03:00
Silent aeebcfe141 Fix Debug - LLVM in VS project files 2020-01-31 11:23:55 +03:00
Eladash 48a847d1b6 Qt: Bugfixes regarding usage of ShowConfirmationBox 2020-01-30 21:49:08 +01:00
Eladash 232c6c3aaf Qt: Display "Reboot With Custom/Global config" on running game 2020-01-30 21:49:08 +01:00
kd-11 c9e35926f5 rsx: Preserve pixel data when splitting sections
- Ironically rhis data leak is caused by trying to fix another type of data leak
2020-01-30 21:07:36 +03:00
Megamouse 8ef69429d8 Add early out to pkg_install 2020-01-30 18:21:18 +01:00
Megamouse 336cd6e33a Qt: Prevent Qt from blocking the explorer during installations 2020-01-30 18:21:18 +01:00
Silent 360e484b08 Fix ellipsis
… is an UTF-8 character and those don't really belong in messages like this anyway
2020-01-30 16:06:00 +03:00
Eladash 92466165f6 Increase Maximum Vblank Rate and Clocks Scale
Allow x30 times the speed of vblank rate + clocks scale of original PS3.
In theory a 60 fps limit game which scales frame limit perfectly with vblank rate can be played at up to 1800 fps with this change.

And:
* Fixed lv2 sleep with Clocks Scaling
* Make these settings dynamicaly adjustable.
* Avoid code duplication
2020-01-29 21:42:41 +01:00
kd-11 1206a5d4b7 rsx: Tweak blit engine heurestics a bit
- Reject writes to RTT if the source data is of unknown origin.
  non-RTT data and only 1 line in length is suspicious and often GPU data like programs or other rendering inputs.
2020-01-29 12:54:06 +03:00
TotalCaesar659 c1d7b46235 Qt: change labels in package installer (#7325) 2020-01-29 00:39:05 +01:00
RipleyTom 795bc5d52b Add mutex guard for s_unfire 2020-01-28 19:16:16 +03:00
Malcolm Jestadt ad8988afd3 Embedded SPU elf patching
- PS3 games include both PPU and SPU code in their PPU executables, so to make patching games that make use of the same SPU libraries easier, we add a system to find and patch them.
- Patches for this system still use SPU LS (Local Storage) addresses despite the fact that we aren't loading anything into SPU LS at this time. The patches are checked against each segment and patched in place.
2020-01-28 02:13:37 +03:00
Ivan 7f07b79c04 Partial revert of #7180
PC is PS
2020-01-27 07:05:18 +03:00
RipleyTom 610a6a1404 Increases number of buffers when buffering 2020-01-27 02:13:30 +00:00
Eladash a7aef22754 ppu: Log SELF header information and CIA of caller HLE functions 2020-01-27 01:21:40 +00:00
Eladash 4e0070f16d Log sys_spu thread group and thread names
Also safely read thread name after relevant error checks passed.
2020-01-26 20:32:10 +00:00
Eladash 7ae679adbe Fix logging of ppu name in sys_ppu_thread_create/rename 2020-01-26 20:32:10 +00:00
Eladash d481c3c7fd cellGameGet/SetParamString: Implement CELL_GAME_ERROR_NOTSUPPORTED 2020-01-26 20:32:10 +00:00
Eladash a9162a3f57 SPU LLVM: Improve approximate FCMGT 2020-01-26 18:37:07 +00:00
Megamouse b341113ad8 Qt: Change some labels 2020-01-26 18:46:04 +01:00
Nick Renieris 1e69de1205 overlays/perf: Graph label tune-up
Place graph text on top, split in 2 lines, center it horizontally.
Also if it's wider than the graph, match up graph's width to it.
2020-01-26 17:55:11 +01:00
kd-11 79216917b3 rsx: Workaround for broken rtt resampling
- Avoids WCB requirement for now to keep res scaling working correctly.
- TODO: Fix this properly
2020-01-26 13:58:48 +03:00
kd-11 698702cd4a vk: Fix DMA data leak
- There still does not exist a ranged flush implementation which is required.
- TODO: Implement this properly
2020-01-26 13:58:48 +03:00
kd-11 1166ae19bb vk: Use appropriate layouts depending on use case when creating new textures to avoid needless barriers 2020-01-26 13:58:48 +03:00
kd-11 44f2cacf7b rsx: Blit engine tuning
- Attempt to identify blit operations that will be flushed immediately
after and just do them on CPU instead if the transformation is trivial.
- If only a single blit section is contributing to an atlas merge op, the
threshold should be 100%. The only acceptable result here is a
truncation.
2020-01-26 13:58:48 +03:00
kd-11 7a275eaa3a rsx: Fix incomplete blit operations getting used as texture inputs
- Raise passing 'score' from 50% to 90% to filter out very incomplete
merge operations.
- Catch unfit sections passing the match test; possible for blit_dst
data but will likely be always harmless. Disabled in release builds by default.
2020-01-26 13:58:48 +03:00
Silent 331c1a394a Qt: Present game removal failure to the user
All the required information was already there,
but UI always reported success
2020-01-25 19:28:52 +01:00
Eladash e4ba096190 VSH: sys_mmapper
* Implement syscalls sys_mmapper_allocate_shared_memory_ext, sys_mmapper_allocate_shared_memory_from_container_ext.
* Implement multi-process shared memory allocations.
2020-01-24 20:08:30 +00:00
Eladash 46df58b662 sys_usbd: Add error_code 2020-01-24 19:25:52 +00:00
Eladash 95ed2ef62e cellGcm HLE: Add error_code 2020-01-24 19:25:52 +00:00
Megamouse 3f076d63e3 HLE: add error checks to cellAudioInGetDeviceInfo 2020-01-23 10:50:55 +01:00
Megamouse 3e8a5c6395 HLE: add some more constants 2020-01-23 10:50:55 +01:00
Megamouse 18f167ddd0 HLE: Fix error checks in cellAudioInRegisterDevice 2020-01-23 10:50:55 +01:00
Maksim Derbasov 1abdee242a small improvement (#7288)
* small improvement

* comments addressed

Co-authored-by: kd-11 <15904127+kd-11@users.noreply.github.com>
2020-01-22 12:28:48 +00:00
kd-11 adcc3e9c4b rsx: Optionally sync on texture read semaphore
- Some games use texture semaphore for zcull sync which is rather bizzare.
  However, it works on realhw as the depth test happens before fragment shader completion
- Due to the high performance penalty incurred by this act, this
behavior is only enabled by the "strict rendering mode" option.
2020-01-21 22:21:51 +03:00
Eladash 949cfa7fdb Fix cellVdecSetFrameRate error check 2020-01-21 16:45:41 +03:00
Eladash fe381b8581 SPU: Add SPU LS to debugger 2020-01-21 16:45:41 +03:00
Eladash 160ddcf86b SPU: Minor FREST bugfix 2020-01-21 16:45:41 +03:00
Eladash f05a3da964 Fix lv2_file::op_write regression 2020-01-21 16:45:41 +03:00
Nekotekina ddda09607d SPU: fixup for STOP 0w0 2020-01-21 16:32:00 +03:00
Eladash 9993df9b8b RawSPU: fix race between spu start and stop
This race could lead to spu status bits indicate RUNNING status, but cpu state being stopped.
Fix it by making sure cpu state is set before spu status.
2020-01-21 14:08:39 +03:00
Nekotekina 98a8eeaac2 SPU: properly support STOP 0x0 instruction 2020-01-20 23:40:10 +03:00
Nekotekina 0f87c6c7c3 Make system config thread-safe (almost) 2020-01-20 21:51:28 +03:00
Nekotekina 0147bc2c72 Fixup shared_cptr, atomic_cptr 2020-01-20 21:51:28 +03:00
Nekotekina 1e7a02badb Implement shared_cptr and atomic_cptr
Limited shared_ptr with atomic support.
Atomic version is only partially implemented.
2020-01-20 16:53:42 +03:00
Nekotekina 3617f12a1e sys_fs: avoid possible out of memory on file reads/writes
Use fixed-sized intermediate buffer.
2020-01-20 16:00:20 +03:00
Nekotekina 63f67c88cc sys_fs: better stub sys_fs_fcntl(0xc0000006)
This syscall does something to classify filesystems by mountpoint.
2020-01-20 16:00:20 +03:00
Nekotekina 1b1b804d7e sys_fs: add /dev_flash mountpoint 2020-01-20 16:00:20 +03:00
Nekotekina 55cb96ab3b sys_fs: fix CELL_EIO condition in cellFsReadWithOffset 2020-01-20 16:00:20 +03:00
Megamouse 5ef3465f65 cellVdec: (experimental) allow AV_PIX_FMT_YUVJ420P 2020-01-20 00:33:25 +01:00
Megamouse 9a27cc9442 cellVdec: improve error checks 2020-01-20 00:33:25 +01:00
Megamouse 858ce014fd VS: fix filter facepalm 2020-01-19 16:38:17 +01:00
Megamouse 4dbad6cce6 fix some random warnings 2020-01-19 16:38:17 +01:00
Megamouse 485b22d664 Qt: fix deprecation warnings 2020-01-19 16:38:17 +01:00
Megamouse 30d5a849e3 Qt: enlarge some tooltip hover areas in settings 2020-01-19 12:40:43 +01:00
MSuih 9ef96e8274 Add pagesteps for some controls
With the snap changes the default pagestep of 1 is ignored because of rounding.
2020-01-19 09:09:36 +01:00
MSuih cbaa8f3329 Add snapping and limit range for wakeup delay 2020-01-19 09:09:36 +01:00
MSuih 807d6cfea4 Add slider snapping 2020-01-19 09:09:36 +01:00
kd-11 22ca2827de rsx: Improve window border detection and clearing
- Improves logic to detect if the frame requires letterboxing and
properly clears the background appropriately.
2020-01-18 19:52:52 +03:00
kd-11 5e0ca4c0c4 rsx: Fixup for missing visuals when framebuffer is larger than requested
display dimensions.
2020-01-18 19:52:52 +03:00
kd-11 48407752a6 formatting: Unify indentation type in the newly added files to tabs 2020-01-18 19:52:52 +03:00
kd-11 bad4d1ff05 rsx: Improve present image scanning
- Adds support for partial (letterboxed) source images by taking insets
into account.
- Bugfix for potential access violation when capturing screenshot on
vulkan
2020-01-18 19:52:52 +03:00
kd-11 7453e46a7c rsx: Refactor out complex present code into separate files
- Also restructures present code to have image lookup in a separate
re-usable function.
2020-01-18 19:52:52 +03:00
Eladash b07b5c9005 Fix sys_spu_thread_initialize for attr->name_len is 0 and attr->name is not null
If name_len is 0 name is empty, in any other case name is not empty (attr->name == nullptr isn't allowed in this case).
Check name_len and option for invalid values as fw.
2020-01-18 18:46:13 +03:00
Eladash 9d15083c61 Fix sys_ppu_thread_create/rename thread name range 2020-01-18 18:46:13 +03:00
Eladash 14b99d9e8b Write nread/nwrite in cellFsWrite/Read regardless of error checks 2020-01-18 15:56:05 +03:00
kd-11 b36b9e4822 vk: Fixup for total number of combined samplers using the dynamic binding structure 2020-01-18 11:17:19 +03:00
kd-11 0a2b6a290d vk: Fixup
- Scaling is not needed for a direct typeless transfer!
2020-01-17 14:31:14 +03:00
Emmanuel Gil Peyrot ca5bc512ae Optimise the SVG logo with svgcleaner
It still contained Inkscape leftovers, which are of no use for anything.
There are no functional changes.

See https://github.com/razrfalcon/svgcleaner
2020-01-17 08:31:00 +01:00
Megamouse 449cbb7281 Qt: use persistent_settings for playtimes 2020-01-17 07:43:10 +01:00
Nekotekina e2512e78b6 sys_fs: always close locked file in sys_fs_close
Syscall returns EBUSY but succeeds nevertheless.
2020-01-17 00:24:07 +03:00
Nekotekina a005090d3d sys_fs: use constant in sys_fs_disk_free 2020-01-17 00:24:07 +03:00
Nekotekina 7fcc49004d lf_fifo: fix UB and fix size()
Simplify internal counter to atomic<u64>.
Make size() return correct difference between push and pop pointers.
2020-01-17 00:24:07 +03:00
MSuih 833fbe015e Use floating point pixel ratio 2020-01-16 23:54:47 +03:00
Eladash c9b0f0e734 SPU: Fix FREST 2020-01-16 23:42:50 +03:00
kd-11 9b34f00241 vk: Optimize image transfers
- Adds the same optimization/simplification steps to complex image
transfer routines. Whenever possible, multi-step transfers are collapsed
into a single operation.
2020-01-16 22:29:26 +03:00
kd-11 82af17beb1 gl: Optimize image operations
- Avoid double transfers where a transfer to a temp image is done
without scaling and then a secondary transfer follows. Combines the two
steps into one whenever possible which can significantly alleviate
bandwidth problems at higher resolutions. Significant speedup, upto 90%
in some cases (PDF, PDF2)
2020-01-16 22:29:26 +03:00
kd-11 47b196e9d0 rsx: Fix uninitialized variable 2020-01-16 17:57:31 +03:00
kd-11 db014d8a58 rsx: Fix section length calculations when generating new blit targets. 2020-01-16 17:57:31 +03:00
kd-11 621fab2ad9 vk: Fix D32S8 interpolation by using integer interpolation instead of floating point
- Interpolating floats is not the same as interpolating their bits!
  Use integer format to interpolate linearly for D32F formats instead of using R32F as intermediary
2020-01-16 11:12:08 +03:00
kd-11 086ecf4ba6 vk: Add some missing image memory barriers causing artifacting on AMD cards
- There needs to be a memory barrier after each step.
- TODO: Optimize scale_typeless_safe function
2020-01-16 11:12:08 +03:00
kd-11 309251ce7a rsx: Touch locked dst memory after blit transfer operations in case it is locked by WCB/WDB 2020-01-16 11:12:08 +03:00
Eladash 9084209cfc Update cellVdecSetFrameRate error checking 2020-01-15 23:29:32 +01:00
kd-11 74ad525566 vk: Fixup for cs_scatter job
- Access to the stencil output has to be atomic as each 'word' is shared among 4 adjacent texels
- TODO: Can be optimized using mirrored buffer views
2020-01-15 21:12:51 +03:00
Eladash 05e5e6058f Add description for rsx wake-up delay 2020-01-15 19:54:23 +03:00
MSuih 556ac1cf22 Add wake-up delay to settings 2020-01-15 19:54:23 +03:00
Eladash 85695c8bac rsx: FIFO wake-up pause control 2020-01-15 19:54:23 +03:00
kd-11 2984300385 vk: Fix invocation alignment to support non-power-of-2 alignment 2020-01-15 15:42:36 +03:00
kd-11 ac4cadf538 vk: Fix word index counting for shuffle tasks 2020-01-15 15:42:36 +03:00
kd-11 175f78f5b3 vk: Lower default compute heap size to 64M
- There is no need to guess and use a large memory footprint as the heap is
now dynamic.
2020-01-15 15:42:36 +03:00
kd-11 3d96fe79cc vk: Implement dynamic sized compute heap
- Implements a dynamically sized compute heap to allow growing up the
size if it is too small.
2020-01-15 15:42:36 +03:00
Eladash 1ccb3c4492 rsx: Verify local memory offset 2020-01-15 13:23:56 +03:00
Eladash 01035d35bd sys_process: Fix sys_process_get_id, add error_code (#7246) 2020-01-14 21:32:41 +03:00
kd-11 8bbda3dedb vk: Restructure command queue flushing behavior to avoid deadlock
- Queueing commands on the offloader is a good idea but unfortunately
page faults can still happen causing a cyclic dependency and eventual
deadlock. Characterized by a vk::wait_for_event timed out error
accompanied by severe hitching.

- Drain the fault-able commands before pushing a submit operation to the
queue. If a fault is in progress, bypass the queue system and submit
raw. Technically this is incorrect but there isn't much that can be
done about it right now.
2020-01-14 14:32:40 +03:00
Megamouse a24514651c Update Appveyor to Qt 5.14 2020-01-14 10:15:35 +01:00
Megamouse 542d2ef8da Qt: smoother batch package installation 2020-01-14 09:27:09 +01:00
Megamouse be2d225d96 sceNpTrophy: deny unlocking of platinum trophies 2020-01-13 22:50:05 +01:00
Eladash 765bd6b6c6 SPU: Optimize gpr reset for MSVC 2020-01-11 22:56:46 +03:00
Nekotekina aeed349a99 sys_fs: adjust permissions for /dev_bdvd
Remove write permissions returned by stat, fstat, etc.
Also make sys_fs_open return CELL_EPERM on write attempt.
2020-01-11 04:48:42 +03:00
Nekotekina 8447d75dda sys_fs: improve sys_fs_lsn_lock
It appears it does nothing only on /dev_hdd0 or /host_root (HOSTFS).
2020-01-11 03:44:52 +03:00
Nekotekina fc6356a74c sys_fs: adjust /dev_bdvd block size
From test.
2020-01-11 03:05:26 +03:00
Nekotekina 41f2ee7b6c VFS: minor change in handling /host_root path 2020-01-11 01:55:36 +03:00
Nekotekina 87cd653c6e sys_fs: improve sys_fs_disk_free
Fix error codes and arg checks.
2020-01-11 01:10:50 +03:00
Nekotekina 7e35fd54a8 sys_fs: improve sys_fs_fcntl(0xc0000002)
Always obtain free space on /dev_hdd0.
2020-01-11 01:09:30 +03:00
Nekotekina 582ee80552 sys_fs: correct block size for /dev_hdd1 2020-01-10 05:24:43 +03:00
Nekotekina 0b4b87f069 sys_fs: fix sys_fs_get_block_size
Don't check file existence on /dev_hdd0.
Relax check on some mountpoints.
Fix CELL_EISDIR error condition.
2020-01-10 05:19:18 +03:00
Nekotekina a8e1afa0af sys_net: fix sys_net_bnet_select arg check (nfds) 2020-01-10 03:21:27 +03:00
Eladash 71df5044fc sys_vm_get_statistics: Write timestamp 2020-01-09 20:43:03 +00:00
kd-11 db5d03c340 vk: Generate dynamic binding table based on the capability of the drivers
- This alleviates constraints imposed on shaders to allow running on some not-so-great platforms.
2020-01-09 15:38:23 +03:00
RipleyTom 4a5559ee65 Add buzz controllers to usb whitelist 2020-01-09 07:51:16 +00:00
Nekotekina 7523416a12 sys_fs: return CELL_ENOTSUP in sys_fs_fcntl(0xc0000006) 2020-01-09 04:15:20 +03:00
Nekotekina 9075208c80 sys_fs: fix logging in sys_fs_get_block_size 2020-01-09 04:15:20 +03:00
Nekotekina d477889763 sys_fs: fix mountpoint detection 2020-01-09 04:15:20 +03:00
kd-11 ef3b0db7d8 vk: Workaround for NVIDIA occlusion query failure
- When using partial results on NVIDIA, a non-zero result is returned even when the draw is fully occluded.
  This, I believe, violates spec which says the partial result shall be between 0 and the final result.
2020-01-08 19:02:45 +03:00
Nekotekina f5cb147f8d sys_fs: improve sys_fs_get_block_size values
Affected other syscalls:
sys_fs_fget_block_size
sys_fs_stat
sys_fs_fstat
sys_fs_fcntl (cellFsGetDirectoryEntries, cellFsGetFreeSize)

For default values:
Return sector_size = 512.
Return 4th arg = 512.
Fod /dev_bdvd:
Sector size = 2048.
Block size = 2048.
2020-01-07 23:16:17 +03:00
Nekotekina 63e669c0cf sys_fs: fix sys_fs_fget_block_size
Return flags via last argument.
2020-01-07 21:55:19 +03:00
Nekotekina 9c54305e10 sys_fs: disable sys_fs_lsn_lock/unlock
According to test, nothing seems to happen.
Disable CELL_EBUSY errors associated with Stream Support API.
2020-01-07 21:55:19 +03:00
Nekotekina f3d52de429 sys_fs: Adjust flags of /app_home mountpoint 2020-01-07 21:55:19 +03:00
kd-11 3f34a0196c overlays/osk: Add linear fade-in/out effect to OSK 2020-01-07 21:31:19 +03:00
kd-11 ecf00be155 rsx: Add color interpolation animation
- Adds color interpolation and modulation pass and refactors the code a
bit. Elements with this pass applied have their color modulated by the
animated color from the pass. Modulation transform is multiplicative.
2020-01-07 21:31:19 +03:00
kd-11 071e73a68e geometry: Allow basic color arithmetic 2020-01-07 21:31:19 +03:00
kd-11 ad845861be video: Remove pointless aspect ratio option
- The auto option is used when requesting the system is works like a
"dont care" specifier to tell the system to use what settings have been
passed in by HDMI EDID or the user TV type setting. Since this option
simulates the "TV type setting", auto makes no sense and is also not
something you can select on a PS3.
- Also adds a few missing checks.
2020-01-07 15:56:54 +03:00
Nekotekina c51779d4d3 Fix format string in log_frame.cpp 2020-01-06 23:44:48 +03:00
Nekotekina 55f9a56e45 Fix sys_tty_write (UTF-8 encoding of literals) 2020-01-06 23:23:04 +03:00
Nick Renieris 5bace118a7 overlays: Redesign animation system (add easing functions, fix bugs)
Instead of speed, direction and distance, the user now specifies
start/end offsets and how much time the transition should take.

Fixes:
- Stuttering caused from framerate estimation.
- An edge case where animations would go over their supposed limit.

Adds:
- The ability to specify arbitrary easing functions for the animations
  - Implemented quadratic ease in and ease out and cubic ease in/out.
- Usage of cubic ease in/out in the trophy notification
2020-01-06 22:42:07 +03:00
Nick Renieris 28770c1580 overlays: Move vertex & vector utility classes to new file 2020-01-06 22:42:07 +03:00
Nick Renieris 192912131e rsx: Update vblank count in LLE mode 2020-01-06 22:42:07 +03:00
Dravonic 94d2f97f27 Multithreaded shader compliation follow-up (#7190)
* Multithreaded load pipeline entries shader compliation stage

Co-authored-by: kd-11 <15904127+kd-11@users.noreply.github.com>
2020-01-06 21:59:59 +03:00
Megamouse 632cc79c54 cellGame: add more checks 2020-01-06 10:47:13 +01:00
Megamouse e7845357e2 sceNpTrophy: unlock platinum trophies 2020-01-05 19:47:31 +01:00
Nekotekina 4450ae0c7a sys_fs: fix CELL_FS_O_APPEND emulation
Don't use fs::append (not capable of).
Fix sys_fs_ftruncate (remove wrong workaround).
2020-01-05 18:15:55 +03:00
Nekotekina 9fc0aec066 sys_fs_stat: fix split file handling
Allow single-file case (consistently with sys_fs_open)
2020-01-05 18:15:55 +03:00
Nekotekina bed2d558a6 sys_fs: implement CELL_EROFS error
Implement lv2_mp_flag::read_only.
Currently only /dev_bdvd is affected.
2020-01-05 18:15:55 +03:00
Nekotekina d5f0957558 Implement lv2_fs_mount_point with mount point flags
Implement some actual mount points
Implement lv2_mp_flag::no_uid_gid
2020-01-05 18:15:43 +03:00
kd-11 7f09def94e rsx/vp: Properly initialize output registers.
- All registers tested on hw show contents to be 0, 0, 0, 1.
  Make default output registers match this pattern.
2020-01-05 18:06:08 +03:00
Silent b88fb43acc Minor cleanup in InstallPkg and InstallPup 2020-01-05 11:01:26 +01:00
Silent e7ddc5187a Add a dialog to Batch PKG Install
Dialog allows users to preview the order in which PKG's will be installed
and allows users to move items around if needed.

Because clicking "Install" on this new dialog acts as a confirmation
and user has a second chance to eyeball what is to be installed,
"Install package X?" dialogs have been removed and instead user
is only notified of success. In case of failure, batch installation
aborts with a descriptive error.
2020-01-05 11:01:26 +01:00
Eladash 872be25ed1 cellFs: Fix cellFsLseek with negative whence 2020-01-04 22:38:53 +03:00
Eladash 9d2c9e5d62 cellFs: Improve cellFsGetDirectoryEntries 2020-01-04 22:38:53 +03:00
Nekotekina 28fb0d1741 Fix lv2_fs_object::name
Recreate path from actual decoded components.
2020-01-04 21:44:03 +03:00
MSuih 5534c9e27c Disable AA for renderers which do not support it 2020-01-04 18:58:33 +01:00
MSuih 049f852a9c Slight cleanup of mousewheel pr
Fixes theoretical uninitialized variable and micro-optimizes scrollwheel stop code
2020-01-04 18:58:33 +01:00
MSuih 69f11d82d1 Change Null microphone to Disabled 2020-01-04 18:58:33 +01:00
Silent f92794360d Qt: Add tooltips to Virtual File System buttons 2020-01-04 17:10:28 +01:00
kd-11 bdb5115c7f rsx/overlays: Improve space usage on trophy dialog
- Slightly increases the size of the trophy dialog and the font size.
  The old dimensions did not work with some libre fonts causing
alignment errors and other problems.
2020-01-04 16:36:49 +03:00
kd-11 3ada97d2d3 rsx/overlays: Implement trophy notification queue
- Allows to display more than one trophy at a time. Trophy notifications
will simply get queued up and displayed at appropriate time.
2020-01-04 16:36:49 +03:00
Nekotekina 49d5ce3ba4 Update glslang.7z 2020-01-04 01:50:28 +03:00
Nekotekina 691a57a4da Add BOM to sys_tty.cpp 2020-01-04 01:32:21 +03:00
Nekotekina 671ae20876 Inject rpcs3_glslang.props to glslang build
Disable new exception format to remove vcruntime140_1.dll dependency.
2020-01-04 01:30:52 +03:00
Nekotekina 7af7e3cec1 Update ffmpeg fork URL 2020-01-03 21:54:37 +03:00
MSuih ca52c1e2d1 Link Bcrypt with ffmpeg 2020-01-03 21:45:09 +03:00
kd-11 31b07fece5 rsx/overlays: Add support for animations
- Adds animation support. This commit adds the base framework and
implements a translate animation used to slide elements around the
screen. This is then used to implement the sliding animation for the
trophy notification.
2020-01-03 20:33:32 +03:00
Megamouse b3ad89cc8b cellSaveData: remove duplicate yield 2020-01-03 14:22:40 +01:00
Megamouse 5e7d25ad35 overlays: refactor shader loading dialogs 2020-01-03 14:22:40 +01:00
Megamouse d94d094a7e overlays: fix non-interactive dialog loops 2020-01-03 14:22:40 +01:00
Megamouse c9aee27d48 VK: remove unused init function declaration 2020-01-03 14:22:40 +01:00
kd-11 d12762414a vk: Change default vertex output value
- Prefer w!=0 to avoid a situation where xyz/w = nan. More of a
theoretical problem, but some calculations break down in such a
situation.
2020-01-03 10:35:53 +03:00
kd-11 7786681954 rsx: Improve MTRSX synchronization
- Properly synchronize DMA transfers when handling RSX pipeline
barriers. Texture read barrier is used to signify completion of DMA
routines and is often used to signal that Cell can overwrite vertex
data!
2020-01-03 10:35:53 +03:00
Megamouse 02ca8f0002 Qt: repaint all related icons for custom configs 2020-01-02 11:35:51 +01:00
Megamouse 7af2ebb6f4 cellSaveData: use errDialog to skip error dialogs 2020-01-02 05:49:03 +01:00
Nekotekina 2c4ecc55af Update ffmpeg 2020-01-02 00:53:22 +03:00
kd-11 c4e59b5115 vk: Clamp depth export in FS
- PS3 matches OGL behavior where writing to the depth export
register results in clamping.
2020-01-01 22:39:20 +03:00
Eladash 4c20881f8f Fixup after #6933 (#7166)
* fixup

* Get rid of obsolute arg in lv2_obj::awake

* nvm ill do this later

* Typo fix of the decade
2020-01-01 17:38:05 +00:00
Ani 341fdf7eb1 rpcs3_version: Bump to 0.0.8 2019-12-31 23:20:49 +00:00
Eladash 7cc6881a57 Implement sceNpTrophyTerm resources deallocation
- Implement sceNpTrophyAbortHandle, if handle is aborted an error code will be returned on its usage.
- Free context & handles in sceNpTrophyTerm.
- Return SCE_NP_TROPHY_ERROR_INVALID_ARGUMENT on invalid context & handles ids.
- Implement id resource shortage error checking.
- Check negative sceNpTrophyGetTrophyInfo trophy id.
- Minor error checking fix for sceNpTrophySetSoundLevel
2019-12-31 22:32:23 +03:00
Eladash 71c4a2a15f Silence some log messages 2019-12-31 22:27:27 +03:00
Eladash 9690854e58 Some cleanup
* Prefer default initializer over std::memset 0 when possible and more readable.
* Use std::format in trophy files name obtaining.
* Use vm::ptr<>::operator bool() instead of comparing vm::ptr to vm::null or using addr().
* Add a few std::memset calls in hle where it matters (or in some places just to document an actual firmware memcpy call).
2019-12-31 22:27:27 +03:00
RipleyTom 5de83e0425 Avoid using vfs::get to let decrypt work on PSN binaries 2019-12-31 22:23:22 +03:00
Eladash c2eb9a583d Implement ps3 application root flags detection 2019-12-31 22:22:06 +03:00
Silent 974bce19ed Use path_append instead of operator + to concat paths
This allows to cleanly prevent double // slashes when appending paths
While this should not be a problem, Windows seems to have problems
with such paths when paths are very long - and preventing this
is trivial enough.
2019-12-31 19:24:19 +03:00
Silent b591633cb9 Fail pkg_install if any files fail to be created
This could happen if eg. paths are too long or some files
could not be overwritten. Until now, installation happily
reported success regardless.
2019-12-31 19:24:19 +03:00
Silent 9e66f36942 Handle ERROR_FILENAME_EXCED_RANGE error on Windows 2019-12-31 19:24:19 +03:00
Silent abfa1114bc Add progress_dialog::SignalFailure()
Turns the taskbar progress bar red, commonly used on Windows
in cases like this to signal failure.
2019-12-31 19:24:19 +03:00
Eladash b30be0fbc1 Update cellVoiceCreatePort syntax 2019-12-31 19:05:06 +03:00
Eladash 9958a83d10 Make spu_channel_4_t::clear() atomic
By removing separate value3 write, fixes a rare race introduced by #6917.
2019-12-31 19:05:06 +03:00
Eladash 3257552227 Minor spu_channel_4_t optimization 2019-12-31 19:05:06 +03:00
Eladash ec9622b513 Fix ppu_patch invalid address check 2019-12-31 18:57:14 +03:00
Eladash 8427af8886 Implement runtime PPU executable code modification via Cheat Manager 2019-12-31 18:57:14 +03:00
Megamouse 93a6e9e4e4 Improve cellImeJp (#7014)
* HLE: add error_code and definitions for cellImeJp

* HLE: add function parameters to cellImeJp

* HLE: improve cellImeJp

* HLE: further improve cellImeJp

* HLE: simplify cellImeJp
2019-12-31 16:02:10 +01:00
Megamouse a5c5927468 cellSaveData: only show dialogs in fixed functions 2019-12-31 14:15:43 +01:00
kd-11 915cf0bae8 vk: Do not leak mapped memory 2019-12-31 13:56:14 +03:00
kd-11 33f7d23571 3rdparty: Update vulkan memory allocator (GPUOpen) 2019-12-30 23:53:10 +03:00
kd-11 bb75174f36 update glslang 2019-12-30 23:53:10 +03:00
Eladash 3e80727f8e Allow unimplemented HLE functions detection by bot 2019-12-30 23:48:39 +03:00
Eladash b59a825e48 Minor fixup after #6894 2019-12-30 23:46:45 +03:00
Eladash 45cff1219c Allow sys_raw_spu_create_tag to be called more than once 2019-12-30 23:46:45 +03:00
Eladash 923cd7ad72 SPU LLVM: rewrite comparison on non-xfloat path of CFLTU, CFLTS
CFLTU on non-xfloat path is accurate as xfloat path now.
* Also optimize FCTIW like FCTIWZ (PPU)
2019-12-30 22:20:34 +03:00
Nekotekina ef8872c0d7 sys_net: fix sys_net_bnet_connect on Windows
Properly write is_connecting variable.
2019-12-29 21:53:43 +03:00
Nekotekina 01b1b21efd sys_net: add a workaround for WSAPoll with timeout
Don't call WSAPoll with empty socket list.
2019-12-29 21:24:13 +03:00
Megamouse 45dc5d9b02 cellSaveData: improve ppu schedule during dialogs 2019-12-29 14:22:58 +01:00
Megamouse c4b4ce46b8 cellSaveData: don't pause apps during dialogs 2019-12-29 14:22:58 +01:00
MSuih 6cfcb7b4f3 Add support for mousewheel movement 2019-12-29 12:45:03 +01:00
kd-11 1b27ccecf8 settings: Add relaxed zcull toggle to advanced tab 2019-12-29 13:49:46 +03:00
kd-11 24cb48971e vk: Fix cb chunk synchronization deadlock 2019-12-29 13:49:46 +03:00
kd-11 e1b734fd12 rsx: Fix linux build 2019-12-29 13:49:46 +03:00
kd-11 ed2bdb8e0c rsx: Zcull synchronization tuning
- Also fixes a bug where sync_hint would erroneously update the sync tag
  even for old lookups (e.g conditional render using older query)
2019-12-29 13:49:46 +03:00
kd-11 fdb638436f rsx: Add toggle for zcull sync behaviour - Adds a relaxed sync mode where ZCULL reports are lazily nudged into flushing and the main core does not actually wait for the event to finish before proceeding - Can drastically improve performance in cases where the game actually does not utilize the report data 2019-12-29 13:49:46 +03:00
kd-11 9f94a6dc11 vk: Refactoring and optimizations to query handling
- Caches query results when looking up report availability to avoid
  entering driver code twice.
- Minor code restructuring
2019-12-29 13:49:46 +03:00
kd-11 55ad9244c0 vk: Switch occlusion pool to FIFO rather than LIFO to avoid hard stall 2019-12-29 13:49:46 +03:00
kd-11 cdd9c12132 vk: Emulate conditional rendering for AMD 2019-12-29 13:49:46 +03:00
kd-11 93895838c7 vk: Implement hw conditional rendering 2019-12-29 13:49:46 +03:00
kd-11 a51395370e vk: Implement multithreaded command submission
- A few nagging issues remain, specifically that partial command stream
  largely caused by poor synchronization structures for partial CS flush
  and also the fact that occlusion map entries wait on a command buffer
  and not an EID!
2019-12-29 13:49:46 +03:00
kd-11 5be7f08965 rsx: Restructure ZCULL report retirement
- Prefer lazy retire model. Sync commands are sent out and the reports will be
  retired when they are available without forcing.

- To make this work with conditional rendering, hardware support is
  required where the backend will automatically determine visibility by
  itself during rendering.
2019-12-29 13:49:46 +03:00
kd-11 8dfea032f2 rsx: Remove deprecated do_method path that has been superceded by c++ inheritance for many years 2019-12-29 13:49:46 +03:00
Megamouse 0b78404337 cellSaveData: get confirmation if not CELL_SAVEDATA_OPTION_NOCONFIRM 2019-12-29 01:19:04 +01:00
Megamouse ff0089bc6f cellSaveData: add more constants 2019-12-29 01:19:04 +01:00
Megamouse bb7848f6e5 cellSaveData: add error dialog to callback result errors 2019-12-29 01:19:04 +01:00
Megamouse 47da907366 cellSaveData: add confirmation dialog for save data interactions 2019-12-29 01:19:04 +01:00
Megamouse 9f106be36d Qt: apply more specific file filters for cache removal 2019-12-28 22:24:06 +01:00
Kravickas f73107b46a rename Searchbar to Search bar in style sheet comments (#7140) 2019-12-28 20:45:12 +01:00
Megamouse 096a08aca1 Qt: fix removal functions after cache compression 2019-12-28 19:44:45 +01:00
Megamouse ef6f565dbd silence some annoying warnings 2019-12-28 15:40:57 +01:00
Nekotekina 70e26eeb45 LLVM: compress PPU cache
Compress PPU modules to .gz (backward compatible with uncompressed cache)
2019-12-27 23:12:16 +03:00
Megamouse e54438d3a7 Qt: add plaintext log and more log options 2019-12-24 13:06:06 +01:00
Megamouse e83dfedff9 stub cellRemotePlay and cellResc and cellStorage 2019-12-24 11:54:32 +01:00
Nekotekina baf9a20c08 Fix UB in to_u8
Possible signed shift overflow.
2019-12-23 22:56:18 +03:00
Nekotekina abe2cc196a sys_net: fixup for race condition
Don't reset is_connecting without checking correctness.
Also cleanup dead WIN32 code.
2019-12-23 22:39:18 +03:00
RipleyTom 90e4fe23c5 WSAPoll workaround 2019-12-23 22:31:09 +03:00
Emmanuel Gil Peyrot 9b77febd10 RSX: Remove two empty cpp files 2019-12-23 00:02:57 +03:00
linkmauve e9c5c6e6bf Move input to its own directory (#7126) 2019-12-22 17:39:42 +01:00
Silent f03cb5c9c0 Qt: Simplify custom titles and notes
By adding some code to notes/titles reading,
it is possible to leverage default value behaviour in QMap value()
which simplifies code a bit.
2019-12-22 15:54:04 +01:00
Eladash c65bb0d267 Avoid shifting by greater/equal than max bits values in ppu_rotate_mask 2019-12-22 04:35:57 +03:00
Markus Stockhausen a36f0497ce Improve approximate xfloat
- Disable denormals for SPU threads
- Add clamping helpers
2019-12-22 01:26:34 +03:00
Nekotekina 3b46c9cb6a SPU LLVM: use FMA with approx xfloat when available
Emulate FMA with double precision if unsupported natively.
2019-12-21 17:53:34 +03:00
Nekotekina 068450d4fe CPUTranslator: detect FMA feature 2019-12-20 21:11:07 +03:00
kd-11 4a5c8c392c appveyor: Update vulkan SDK version to 1.1.126.0 2019-12-20 16:57:01 +03:00
kd-11 fff1007d9a cmake: Fix vulkan on X11 2019-12-20 16:57:01 +03:00
Eladash db4041e079 Implement rounded_div
Round-to-nearest integral based division, optimized for unsigned integral.
Used in sceNpTrophyGetGameProgress.
Do not allow signed values for aligned_div(), align().
2019-12-20 14:47:04 +03:00
Emmanuel Gil Peyrot e30173a835 rsx: Make X11 optional on Linux
This makes it possible to build rpcs3 on a pure Wayland system, without
the Xlib installed.
2019-12-20 10:48:03 +00:00
Eladash 6dd4003154 Fix cellNetCtlNetStartDialogUnloadAsync 2019-12-20 10:20:32 +01:00
Silent 95f363c174 Qt: Look for custom game titles when filtering via Search 2019-12-20 00:19:13 +01:00
Megamouse 7fcef227af sceNpTrophy: minor optimizations 2019-12-19 18:15:22 +01:00
Megamouse 6388c98073 Qt: add platinum relevance to trophy_manager_dialog 2019-12-18 15:41:15 +01:00
Megamouse 91ad3d371f Qt: minor optimizations in trophy_manager_dialog 2019-12-18 15:41:15 +01:00
Eladash 8d955cfb8c Hotfix sceNpTrophyGetTrophyUnlockState 2019-12-18 13:07:55 +01:00
Eladash e380a8c279 Fix sceNpTrophyGetGameProgress (#7103)
* Fix sceNpTrophyGetGameProgress

Was missing multiplication by 100.

* apply requested changes

* Enforce round-to-nearest rounding mode
2019-12-17 22:43:00 +00:00
Eladash 3efd5f360c PPU DisAsm: Again 2019-12-15 19:55:23 +03:00
Megamouse 725761f282 Qt: move firmware settings to the advanced tab 2019-12-14 15:15:49 +01:00
Megamouse 97fb08935e Qt: Use QT_ENABLE_HIGHDPI_SCALING starting with Qt 5.14 (#7090)
* Qt: Use QT_ENABLE_HIGHDPI_SCALING starting with Qt 5.14

* Qt: Add QT_SCALE_FACTOR_ROUNDING_POLICY starting with Qt 5.14

* Qt: remove style path for Qt versions below 5.11 (5.11.0 is min version)

* Qt: Fix dpi-rounding cli arg

* Qt: use rounding policy PassThrough as default dpi scaling policy

* Qt: add missing Qt Version check (facepalm)
2019-12-14 12:48:34 +01:00
Jayveer 364ba95d26 Updated sys_net to use WSAPoll on win32 as this fixes connection issues
in Metal Gear Online. Also aligns with implementations on non win32
platforms
2019-12-13 21:52:29 +03:00
Nekotekina e3e97da7bb LLVM: remove cascadelake workaround and update LLVM fork
VPTERNLOG fix may be necessary.
2019-12-13 16:19:08 +03:00
Nekotekina 321f7e7197 Fix missing-braces warnings 2019-12-13 03:21:43 +03:00
Nekotekina dbece4d00f LLVM: add workaround for cascadelake CPU
Need to investigate it later.
2019-12-13 02:11:21 +03:00
Nekotekina 69e81da57b Fix some warnings 2019-12-13 02:08:52 +03:00
kd-11 73236efe58 vk: Remove some outdated code (#7060) 2019-12-12 16:29:55 +03:00
Eladash 6a926daee7 rsx: Delay FIFO recovery point creation if is in in_begin_end scope (#7080) 2019-12-12 15:38:56 +03:00
Megamouse 038694ae28 cellGem: add some explanatory comments 2019-12-10 23:20:13 +01:00
Megamouse 691c7cde67 cellGem: cleanup some spacings 2019-12-10 23:20:13 +01:00
Zangetsu38 af8ebc76e2 Modules/cellGem: Implement pos, quat and handle_pos in Gemstate for mouse.
Modules/cellGem: Fix name for gem_image_state.
Modules/cellGem: Implement projectiion(x/y) in gem_image_State for mouse.
Modules/cellGem: Add cross, triangle, circle and start with use middle click for mouse.
Modules/cellGem: Refactor global code.
Modules/cellGem: fix some warning with initializing value.
2019-12-10 23:20:13 +01:00
VelocityRa ac2b0c9752 PSMove: Rudimentary mouse emulation. 2019-12-10 23:20:13 +01:00
Eladash 7260af032e rsx: Ignore or recover from unknown primitives
This also fixes a bug when recovering FIFO or creating such recovery point inside in_begin_end == true scope.
2019-12-11 00:11:12 +03:00
Eladash 6a4ba9d562 Fix warning about truncation 2019-12-07 10:22:11 +01:00
Eladash a5b3b0df40 sceNpBasicAddBlockListEntry: Add connection check 2019-12-07 10:22:11 +01:00
Eladash 963d5fb981 sceNpBasicAddPlayersHistory(Async) fixes
* Check if count == 0.
* u64 typos fixed.
* description won't be checked if null.
* Check each npid provided, not only the first.
2019-12-07 10:22:11 +01:00
Eladash 5142d7c011 Minor fixup after #7068 2019-12-06 19:59:29 +03:00
Eladash 5631382623 sys_spu: Fix SPU Thread Id
* Removed wrong code in sys_spu_thread_group_terminate.
* SPU Thread ID is accurate, including 5th thread id "rule".
* Fixed possible use-after-free access of spu_thread::group member.
* RawSPU ID management simplified.
2019-12-06 19:59:29 +03:00
Eladash e050dcbc52 Move size write in cellGamePatchCheck 2019-12-05 23:31:46 +03:00
Eladash 98970884c8 Make sys_tty accurate in release console mode, message saving in log 2019-12-05 23:31:46 +03:00
Eladash 401813f962 Revert part of #7044
oops.
2019-12-05 23:31:46 +03:00
Eladash a2c69e4a2e sys_spu_thread_group_dis/connect_event Improvements 2019-12-05 23:31:46 +03:00
Eladash c3862b74b1 Compilation fix for NetBSD (#7067) 2019-12-05 11:28:00 +03:00
Nekotekina 835892aa51 C-style cast cleanup VII 2019-12-05 02:10:15 +03:00
Nekotekina d2fd3c6bc4 Commit 377e7d2a73 2019-12-04 21:32:08 +03:00
Nekotekina 377e7d2a73 C-style cast cleanup VI 2019-12-04 17:56:22 +03:00
kd-11 d973835bef ds4: Cleanup
- Simplify helpers to read integers HID feature report buffers. They're
simple LE values.
- Improve calibration format detection by just checking the returned
values and not relying on bluetooth check. PS4 does not seem to do it
this way as it accepts 3rd party controllers with inverted format. The
only other request 0xA3 seems unrelated to device capabilities.
2019-12-04 13:48:39 +03:00
kd-11 6ae8fd0d34 ds4: Add workarounds for some 3rd party controllers 2019-12-04 13:48:39 +03:00
scribam 2eaaf5b132 vk: Add sampleRateShading to the list of device enabled features 2019-12-04 12:59:38 +03:00
Nekotekina 185c067d5b C-style cast cleanup V 2019-12-03 17:23:00 +03:00
Megamouse 46ca39ec4d Add Emu.HasGui() to properly hide Qt dialogs in no-gui mode 2019-12-03 09:12:51 +01:00
Megamouse 84f9911163 Qt: skip in-game Qt dialogs if no-gui was used 2019-12-03 07:59:10 +01:00
Megamouse d73f04da13 cellSaveData: avoid nullptr if there is no dialog 2019-12-03 07:59:10 +01:00
Nekotekina bf11a28fb5 C-style cast cleanup IV 2019-12-01 22:12:33 +03:00
Eladash d45fbc331c Fix error checking of cellGameContentPermit 2019-12-01 21:11:43 +03:00
Eladash 1a6e8e20dc cellGameBootCheck Improvements
* size->sysSizeKB returns 4kb. (hw tested, unknown if in different situations this value changes)
* Return CELL_GAME_ERROR_BROKEN on failure to init. (hw tested)
* Write to dirName only when type is HDD game. (REd)
* Don't write to *size on errors, make it happen after *attributes and *type writes. (REd)
2019-12-01 21:11:43 +03:00
Eladash 37c220af72 Fix sceNpTrophyRegisterContext hack 2019-12-01 21:11:43 +03:00
Nekotekina 28eacc616a C-style cast cleanup III 2019-12-01 00:32:44 +03:00
Nekotekina ad9c9f0183 C-style cast cleanup II 2019-11-30 18:17:45 +03:00
Nekotekina 5b9df53c13 C-style cast cleanup (partial)
Replace C-style casts with C++ casts.
2019-11-29 00:35:23 +03:00
Eladash 8bd52c9843 Fix sys_spu_thread_un/bind_queue queue existence check 2019-11-28 01:54:45 +03:00
Eladash 9cc3838e49 Fix possible division by zero in sys_spu_thread_write/read_ls 2019-11-28 01:54:45 +03:00
Eladash b8220ec12f Stub sys_spu_thread_group_log 2019-11-28 01:54:45 +03:00
Eladash 038c6bb234 Avoid calling lv2_obj::sleep sequantially in sceNpTrophyRegisterContext
Also fix a correctness issue with queued.
2019-11-27 21:41:40 +01:00
Eladash 3b06214f88 Decrease memory stats in cellGemInit if needed
Also:
* fix state reset of cellGem
* Check max_connect == 0 in cellGemInit
2019-11-27 21:41:40 +01:00
Megamouse d91f8193b0 settings: enable dynamic reload of some emu values 2019-11-27 10:34:03 +01:00
Megamouse f2b530823b overlays: add dynamic switch for perf overlay 2019-11-27 10:34:03 +01:00
Eladash b5f6b27f86 cellSysutilGetSystemParamString improvements
- Check bufsize, must match to max string length
- Add missing cases for ParamId 0x1008, 0x1011, 0x1012, 0x1024
- Set actual current username in buffer on id == ID_CURRENT_USERNAME
2019-11-26 20:21:33 +00:00
Nekotekina 2290c389d6 vm: implement vm::try_access, vm::ptr::try_read/write 2019-11-26 00:12:45 +03:00
Nekotekina 1b9a3e6077 SPU: internal refactoring, add spu_program
Use struct (spu_program) instead of std::vector<u32>.
2019-11-26 00:01:38 +03:00
Nekotekina 4caf747729 SPU LLVM: detect mpy32 pattern 2019-11-26 00:01:38 +03:00
scribam e735346d26 cmake: Set LLVM_INCLUDE_BENCHMARKS to OFF 2019-11-25 23:08:37 +03:00
scribam f29709105f cmake: Add post-build commands for Windows 2019-11-25 23:08:37 +03:00
Eladash 70ace4bb83 Fix cellAudioCreateNotifyEventQueue queue depth
TODO: global sdk value used is set in cellAudioInit, which means sdk var value is 0 before it's initialization, currently safely set queue size to 8 in this case.
2019-11-25 09:04:56 +00:00
Eladash 17ae70b935 Implement cellAudioCreateNotifyEventQueueEx 2019-11-25 09:04:56 +00:00
Eladash 40db205f5d Fix cellVoiceCreatePort 2019-11-25 09:04:56 +00:00
Eladash 1233c48d92 cellAudio: Implement event source 2019-11-25 09:04:56 +00:00
Eladash f4ac8a2e0c cellVoice: Fix NOT_INIT checks, implement 0 event source handling 2019-11-25 09:04:56 +00:00
Eladash c9066e7236 cellVoice Update
* Use shared lock wherever possible.
* Fix cellVoiceSetVolume.
* Fix error logging of cellVoiceCreateNotifyEventQueue, cellAudioCreateNotifyEventQueue
2019-11-25 09:04:56 +00:00
Eladash fd433779bb sys_prx: Tiny improvement 2019-11-25 09:04:56 +00:00
kd-11 8ca53f9c84 rsx: Remember to min-max the anchor indices of a polygon or triangle fan 2019-11-24 19:01:57 +03:00
kd-11 429a76a140 rsx: Remove redundant check 2019-11-23 16:11:18 +03:00
Nekotekina f31233b822 sys_net: don't report SYS_NET_EWOULDBLOCK in blocking syscalls 2019-11-22 18:56:40 +03:00
Nekotekina 5c55d4f2fe sys_net: format both negative and positive sys_net_error values 2019-11-22 18:56:13 +03:00
Eladash 6ad8b0a443 sys_net: Add error_code 2019-11-21 22:51:22 +03:00
Eladash 8d5ef2c7b9 Clamp volume in cellAudioAdd/2c/6c/Data
Same as firmware, NaN and Inf are flushed to zero.
2019-11-20 21:58:43 +03:00
Markus Stockhausen cd6b6c8a4f Lightweight putllc() for non-TSX if no data changed
This replaces the totally messed up PR #6728

Some games make heavy use of getllar() & putllc() without even changing data.
In this case avoid unneccesary heavy locking of the PPU threads on non-TSX
hosts.
2019-11-19 18:10:29 +03:00
Megamouse 223d1473b0 Qt: remove unused rsx debugger code 2019-11-19 16:04:03 +01:00
kd-11 41e7d2aa0a rsx: Select correct image aspect for blit engine targets. 2019-11-19 13:18:15 +03:00
kd-11 fd751e3e7b rsx: Improve blit format mismatch detection 2019-11-19 13:18:15 +03:00
kd-11 41c3180276 rsx: Fix invalid format checks for DMA sections which are typeless 2019-11-19 13:18:15 +03:00
Megamouse 2152a77005 HLE: stub cellCrossController some more 2019-11-18 23:42:49 +01:00
Megamouse 41505117a7 HLE: stub cellCelpEnc and cellCelp8Enc 2019-11-18 23:42:49 +01:00
Megamouse ca59ab59e9 HLE: add even more error_code 2019-11-18 23:42:49 +01:00
kd-11 9dab0575fa rsx: Add missing format check for the RTV<->DSV transfer case
- TODO: Rewrite resource handling routines
2019-11-18 13:17:00 +03:00
kd-11 4a0e1c79ed rsx: Improve format validation for blit engine
- Check all possible cases where format mismatch is possible.
- Warn if a slow path is going to be taken. Should help with future
optimizations.
2019-11-18 13:17:00 +03:00
kd-11 c415578e79 vk: Clamp buffer row length to never be less than declared width
- Fixes some games with broken textures
2019-11-18 13:17:00 +03:00
kd-11 2408922806 rsx: Do not ignore clamping for some routines that do not have implied range 2019-11-18 13:17:00 +03:00
kd-11 c10aa360b1 rsx: Remove more deprecated methods 2019-11-18 13:17:00 +03:00
JohnHolmesII e53be99e1a Fix typo in update manager 2019-11-18 07:33:53 +01:00
Nekotekina db4e201fee SPU LLVM: use updateGlobalMapping instead of addGlobalMapping
...Where appropriate.
Fix to incorrect addGlobalMapping usage.
Also use clearAllGlobalMappings at compilation start.
2019-11-17 22:15:59 +03:00
Nekotekina f30b81f8b0 SPU LLVM: change patchpoint name generation
Allow 2 or more patchpoint from a single address.
2019-11-17 22:07:08 +03:00
Niram7777 2de3c06781 Qt: Avoid Start/StopPlaytime when installing firmware. 2019-11-17 10:57:21 +01:00
Megamouse 9a8c59a1ff Qt: rename pad_info to pad_device_info for ODR 2019-11-16 16:11:40 +01:00
Megamouse 0b28f0fa14 HLE: add more error_code 2019-11-16 10:38:56 +01:00
Eladash 0f2adab05f Disallow NaN in cellVideoOutSetGamma 2019-11-16 11:21:10 +03:00
Eladash 610463a425 cellSysutilGetSystemParamInt(ID_PAD_RUMBLE) changed to return ON status 2019-11-15 20:17:51 +01:00
Eladash da12969898 cellSysUtil: Add a few CELL_SYSUTIL_ERROR_VALUE checks 2019-11-15 20:17:51 +01:00
Eladash 894658f14c cellSysutil: Log error code 2019-11-15 20:17:51 +01:00
Eladash 25382a7dae Restore cellPadSetActDirect's reserved error check 2019-11-15 20:17:51 +01:00
Megamouse a17a5a76a0 overlays: avoid division by zero 2019-11-15 14:53:18 +01:00
Megamouse fb96047d2f overlays: add settings for overlay graphs 2019-11-15 14:53:18 +01:00
Megamouse dd1707bd46 overlays: fix center options when graphs are shown 2019-11-15 14:53:18 +01:00
Megamouse d6b0361a02 overlays: perf_metrics_overlay to seperate header
this is done to prevent severe conflicts with upcoming changes
2019-11-15 14:53:18 +01:00
Megamouse b84fd29abc settings: fix performance overlay margin checkbox 2019-11-15 14:53:18 +01:00
Anuskuss 7e31c30133 Intel iGPU needs workaround on Windows 2019-11-15 12:08:16 +03:00
Nekotekina fcc841eaa1 LLVM: Add workaround for znver2 2019-11-14 19:09:34 +03:00
RipleyTom 8e39c778a1 Cheat engine 2019-11-13 00:53:31 +03:00
Nick Renieris cc59d319e1 overlay: Performance graphs 2019-11-12 20:43:09 +01:00
msuih 1af948a7d4 Fix bitfield warning 2019-11-12 21:36:29 +03:00
Emmanuel Gil Peyrot 0ff5938b1f Utilities: Remove unused headers
AtomicPtr.h has never been used since its introduction in
da7472fe81.

Same for Interval.h, since 4fedf5749e.

event.h’s last usage got removed in
70e2873e69.
2019-11-12 19:51:57 +03:00
Nekotekina abbf3c4d16 SPU LLVM: add SPU profiling to compilation thread
Run another thread to collect profile data from SPU threads.
Use this data to prioritize compiling hot spot SPU blocks.
Implement stx::init_mutex::wait_for_initialized() helper.
2019-11-11 23:13:13 +03:00
Eladash a21d7def33 SPU: Don't break blocks on DSYNC instruction in Safe mode (#6731) 2019-11-11 23:12:21 +03:00
kd-11 8234bdb8f0 vk: Check for heap change events after a grow to avoid spec violations
- Avoid referencing the old buffer in stale views. Status can be set
globally if requested during heap creation.
2019-11-10 17:53:12 +03:00
kd-11 5968427a2f vk: Initialize queries before use
- The spec does not guarantee that queries are initialized. In fact, it
now says all queries must be reset before they are used for the first
time.
2019-11-10 17:53:12 +03:00
kd-11 8ea9bc9874 vk: Reduce memory allocation sizes of default heaps
- The heaps will grow as desired, no need to overallocate to cater to
the most resource-hungry games
2019-11-10 17:53:12 +03:00
kd-11 0a32d478df vk: Enable auto-growing of the data heaps for the performance case 2019-11-10 17:53:12 +03:00
kd-11 357e0d2097 vk: Implement explicit runtime flags to manage events like heap sync 2019-11-10 17:53:12 +03:00
kd-11 f359342721 rsx: Implement mutable ring buffers with grow support 2019-11-10 17:53:12 +03:00
kd-11 5f39a594ac rsx: Clean up some unused legacy methods unnecessary after d3d removal 2019-11-10 17:53:12 +03:00
Eladash 3f3ce53c67 Fix string length check in cellGameThemeInstall
vm::ptr::size() returns sizeof type instead.
2019-11-10 14:32:15 +03:00
Eladash e8cc9d1f52 Loader: Fix default sdk version
Note: sdk version checks are unsigned checks, such value results in always true.
2019-11-10 14:32:15 +03:00
Eladash 1986944c61 Implement sysutil_check_name_string
* Fully implement 3, 4, 16 and 17 error checks of cellSaveData.
* cellSysCacheMount restrict characters in ID.
* cellHddGameCheck restricts characters in dirName.
2019-11-10 14:32:15 +03:00
Ivan 7519e36317 Merge pull request #6970 from linkmauve/switch-gsl-tcb
Switch gsl::span from GSL to tcbrindle’s implementation
2019-11-09 23:07:58 +03:00
Emmanuel Gil Peyrot 56f82d2701 rsx: Wrap gsl::span definition into Utilities/span.h 2019-11-09 20:00:50 +01:00
Emmanuel Gil Peyrot f29fcc10cf s/GSL/span/ in XML files too
This is completely untested, as I can’t run Visual Studio.
2019-11-09 19:30:06 +01:00
Emmanuel Gil Peyrot 4401ad6747 Fix CI 2019-11-09 19:30:06 +01:00
Emmanuel Gil Peyrot 7e4d142682 Remove GSL as we don’t use it anymore 2019-11-09 19:30:06 +01:00
Emmanuel Gil Peyrot f76720ceb0 Remove extraneous ::narrow<int>() calls
GSL’s gsl::span didn’t use the correct type for its index_type, which is
why they were needed.
2019-11-09 19:30:06 +01:00
Emmanuel Gil Peyrot 72cdf0b04c Replace gsl::span’s implementation with tcbrindle’s
This implementation optimises correctly on all relevant compilers,
unlike GSL’s which gave extremely slow code on any compiler other than
MSVC.

Supersedes #6948.
2019-11-09 19:30:06 +01:00
Emmanuel Gil Peyrot 25144caf12 Add tcbrindle’s span implementation 2019-11-09 19:30:06 +01:00
Nekotekina a83cbf63c6 Remove gsl::finally only use 2019-11-09 19:30:06 +01:00
Emmanuel Gil Peyrot ef368c5171 rsx: Replace gsl::byte with C++17’s std::byte 2019-11-09 19:30:05 +01:00
Nekotekina 6ea02c23a0 Fix SSSE3 usage in PPUInterpreter.cpp
Fixup
2019-11-09 20:39:56 +03:00
Nekotekina 1cca00ec3a Fix SSSE3 usage in PPUThread.cpp
Remove legacy ifdefs, allow SSSE3 usage in AppImages.
2019-11-09 20:10:59 +03:00
Nekotekina 191675bf31 Remove gsl::finally only use 2019-11-09 19:11:01 +03:00
Nekotekina 0d629743ca cellSysCache: poison opened cache files on cache clear
Make opened files return CELL_EIO after cache clear.
2019-11-09 14:50:41 +03:00
Nekotekina 73ee6abb0f cellSysCache: return /dev_hdd1 root
It simplifies internal logic a little.
2019-11-09 14:43:19 +03:00
Nekotekina e0bc276205 cellSysCache: keep mounted /dev_hdd1 after exitspawn 2019-11-09 14:43:19 +03:00
kd-11 7072489a6e rsx: Implement point sprite coordinate generation
- When the point sprite flag is set, overrides the input similar to the
2D mask. The returned X and Y values are always the gl_PointCoord values
for the fragment.
- Stacks with the 2D mask to override the z and w coordinates.
2019-11-09 12:50:53 +03:00
Nekotekina 81c61e230f Use std::from_chars in Emulator::SetUsr 2019-11-08 19:27:11 +03:00
Nekotekina 218758183d Avoid exceptions in cfg::try_to_int64 and cfg::try_to_enum_value
Use std::from_chars plus minimal hex prefix support.
2019-11-08 19:27:11 +03:00
Nekotekina ccac9d4777 Remove throwing and catching cpu_flag::stop
Since there is spu_runtime::g_escape function now.
2019-11-08 19:27:11 +03:00
Nekotekina e74a311931 GDB: remove wrong_checksum_exception 2019-11-08 19:27:11 +03:00
Nekotekina 587ae17aa2 Simplify fmt::throw_exception
Gradual exception deprecation: disallow choosing exception type.
However, the function itself can remain here forever.
2019-11-08 19:27:11 +03:00
Nekotekina b338014639 Don't throw in operators _u16 _u32 _u64 2019-11-08 19:27:11 +03:00
Nekotekina 742bd633d8 Remove fs::file::set_delete
Since it's Windows-only functionality with limited use.
2019-11-08 19:27:11 +03:00
Nekotekina d6087978b5 Fix vfs::host::unlink (Win32)
Use fs::get_virtual_device properly.
2019-11-08 19:27:11 +03:00
kd-11 63673b1a9f rsx: Implement full color remap for the D24S8->ARGB8 converter 2019-11-08 19:11:59 +03:00
Nekotekina 24fdd24808 Fix vfs::host::remove_all
Separate WIN32-specific logic.
Don't call fs::remove_all on WIN32 path.
2019-11-07 21:21:21 +03:00
RipleyTom 4ff6acf6ba Fixes evdev settings & multiple pads handling 2019-11-07 15:02:01 +03:00
Eladash af0bd7136d PPU Disasm: Fix RC OE enabled instructions view 2019-11-07 14:09:23 +03:00
kd-11 8d1505752f rsx: Validate depth test setup to avoid address contention 2019-11-07 11:32:44 +03:00
Megamouse 77ac875b0b Qt: adjust custom context menu positions 2019-11-06 19:26:29 +01:00
Nekotekina 03b9ee27c5 Virtualize cellSysCache
Allows caches from multiple games to coexist.
Also change the way of handling cache IDs (file-less).
2019-11-06 14:12:43 +03:00
kd-11 508ffcb775 vk: Compute kernel fixups
- Adhere to workgroup count limits as exposed by the GPU vendor.
  They already execute properly even when going beyond the limits but this removes validation noise.
- Fix invocation counts for deswizzle kernel. The count was incorrect if blocksize was not 4, causing a bunch of useless work to be done.
2019-11-05 22:07:22 +03:00
kd-11 99d71fdc2a vk: Implement layer batching for the GPU swizzle decoder
- Handles all LODs per layer meaning cubemaps are now fully handled in 6 passes instead of 6 * (log2(width)) passes.
- Handles all LODs of a 3D texture in one pass as well.
- The improvements do warrant dropping down the number of allowed compute invocations a bit
2019-11-05 22:07:22 +03:00
kd-11 7a0b94f343 vk: Minor compute optimizations
- Remove use of uniform buffers for compute static data. Use push
constants instead.
- Minor touchups to the deswizzle code to avoid redundant data copies.
2019-11-05 22:07:22 +03:00
kd-11 1266b63135 vk: Enable gpu deswizzling 2019-11-05 22:07:22 +03:00
kd-11 9cd3530c98 rsx: Set up framework for hw deswizzle 2019-11-05 22:07:22 +03:00
Nekotekina 531afe0f97 atomic2: fix pessimisations 2019-11-04 22:53:26 +03:00
Nekotekina ec25deed2b atomic2: fix corruption bug 2019-11-04 22:52:45 +03:00
Nekotekina be306afae9 atomic2: add TSX path for multi_cas 2019-11-04 22:52:24 +03:00
Nekotekina 1f9250e39a Linux: disable update check at startup if not running from AppImage 2019-11-04 22:52:03 +03:00
Nekotekina da135fbfcc SPU LLVM: Disable "special" branch patchpoints 2019-11-04 22:52:03 +03:00
kd-11 57d3c9e171 rsx: Take empty queries into account for engines that spam report reads.
- Some games will spam the report queue with requests but have zpass
statistics enabled.
2019-11-04 18:48:41 +03:00
kd-11 2a8f2c64d2 rsx: Implement report transfer deferring
- Allow delaying report flushes triggered by image_in or buffer_notify
- When the report is ready, all the delayed transfers will automatically
be done.
- TODO: Make this configurable?
2019-11-04 18:48:41 +03:00
kd-11 3e0f9dff4d vk: Improve zcull synchronization
- Use zcull sync hints more aggressively
2019-11-04 18:48:41 +03:00
kd-11 fe3c290d03 vk: Reimplement occlusion result reading
- Implement partial result reads
2019-11-04 18:48:41 +03:00
kd-11 51e0eaaddc rsx: Implement backend notification for upcoming zcull reads 2019-11-04 18:48:41 +03:00
kd-11 df63de8f16 rsx: Allow u32 restart index with full index width 2019-11-04 16:56:34 +03:00
kd-11 6b3af09fa5 vk: Improved crash message for missing MSAA features 2019-11-04 16:56:34 +03:00
ipkpjersi 8ed45a0553 Qt: Fixed plurals of time played strings (#6937) 2019-11-03 21:22:23 +01:00
Silent e1638b9326 Fix context menu showing twice in Game List 2019-11-03 20:13:20 +01:00
Eladash 7475be99ab lv2: Check ipc_key value if object is process shared 2019-11-03 14:59:47 +03:00
Eladash d2f3ada312 bf_t: Remove <limits> lib dependency 2019-11-03 14:59:47 +03:00
eladash 33fe0fdd6b Add missng check for key in cellAudioSetNotifyEventQueue
key is checked once according to hw test.
2019-11-03 14:59:47 +03:00
Megamouse 3be92ec52b Qt: implement time played and last played 2019-11-02 12:48:12 +01:00
RipleyTom 35c0c05707 Stop autoupdater renaming file across devices 2019-11-02 12:31:38 +03:00
Nekotekina f75661bfae Fix ALSA and PULSE building on Linux (workaround) 2019-11-01 20:41:16 +03:00
Nekotekina 35249d2578 Implement multi_cas with atomic2 type
stx::atomic2 is a "fat atomic" for use with multi_cas.
stx::multi_cas is minimal transaction routine.
2019-11-01 20:41:16 +03:00
kd-11 bbed791ee0 vk: Add explicit support for identity image views
- Allows bypassing all remap shenanigans to make some operations that
rely on the raw image to work correctly.
2019-11-01 19:35:46 +03:00
Emmanuel Gil Peyrot 03a2d36c7e Audio: don’t build ALSA, PulseAudio or FAudio backends when disabled 2019-10-31 15:46:20 +03:00
kd-11 63bbf11a76 vk: Add video out calibration pass
- Adds gamma correction and RGB range filters to output to match PS3
2019-10-31 14:43:24 +03:00
kd-11 78aefe5b5e rsx/overlays: Add support for other primitive types other than triangle_strips 2019-10-31 14:43:24 +03:00
Nekotekina e3e7051ed3 Minor optimization in BufferUtils.cpp
Don't use PSHUFB for horizontal operations.
Utilize PHMINPOSUW to compute max as well:
 + sse41_hmin_epu16
 + sse41_hmax_epu16
2019-10-30 18:52:34 +03:00
Nekotekina b1968769b7 Minor cleanup in BufferUtils.cpp
Replace inline asm with intrinsic using target attribute trick.
2019-10-30 17:53:51 +03:00
linkmauve cfd5cf6bdb Optimise primitive_restart::upload_untouched() (#6881)
* rsx: Optimise primitive_restart::upload_untouched() with SSE4.1

This optimisation is only applied when skip_restart is false.

I’ve only tested the u16 codepath, as it is the one used in NieR.

In some very unscientific profiling, this function used to take 2.76% of
the total frame time at the save point of the port town, it now takes
about 0.40%.

* rsx: Mark all SSE4.1 functions with attributes on gcc and clang

This assures the compiler we will take care of only calling these
functions after having checked that the CPU does support these
instructions.

* rsx: Add an AVX2 implementation of primitive restart ibo upload

* rsx: Remove redefinition of SSE4.1 instructions

Now that clang is aware that our functions are compiled with SSE4.1, it
lets us generate this code using its intrinsics.

* rsx: Optimise vector to scalar conversion

This is done using minpos and srli intrinsics and generate less code
than before.

Thanks Nekotekina for the suggestion!
2019-10-30 16:42:44 +03:00
kd-11 35794dc3f2 vk: Add checks for alphaToOne support
- This feature is very rarely used, as alphaToCoverage is commonly used as a replacement for blending, not in addition to it.
2019-10-30 01:06:28 +03:00
Eladash f41f5054f7 sys_spu Fixup after #6864 2019-10-29 23:13:16 +03:00
kd-11 eda09489b2 vk: Optionally ignore depth bounds testing on hardware that does not
support it.
2019-10-29 20:03:54 +03:00
kd-11 7a5c20ef85 vk: Minor spec touchups
- Simplify active instance management. While multicontext support will
be required in future, this is better done with multiple logical devices
rather than multiple instances.
- Destroy the WSI surface on exit
- Enable depthBoundsTest explicitly. TODO: Properly check for supported
features.
2019-10-29 20:03:54 +03:00
kd-11 aa3eeaa417 rsx: Separate subresource_layout:dim_in_block and
subresource_layout::dim_in_texel

- These two are not always linked when working with compressed textures.
The actual texels extend past the actual size of the image if the size
is not aligned. e.g if height is 1, the real height is 4, but its not
possible to determine this from the aligned size. It could be 1, 2, 3 or
4 for example.
- Fixes image out-of-bounds writes when uploading from CPU
2019-10-29 20:03:54 +03:00
Eladash b99992d570 sys_spu: Fix SNR and Inbound Mailbox state reset
Also remove bugged ESTAT check at sys_spu_thread_write_spu_mb.
2019-10-29 18:34:28 +03:00
Eladash 739f68271f SPU LLVM: fix MFC CMD enqueuing 2019-10-29 18:16:08 +03:00
Eladash f88a135b5b Fix lv2_config_service_event event id reset 2019-10-29 18:15:26 +03:00
Eladash 114b9498bf Remove unjustified use of idm::import_existing in sys_config 2019-10-29 18:15:26 +03:00
Eladash 8b6383b489 lv2: Correct alignment of some attributes structs 2019-10-29 18:15:41 +03:00
Eladash f7c177d79e Kernel Explorer: Don't hardcode endianness in name64 2019-10-29 18:15:26 +03:00
Eladash 3e4cbf67f6 sys_rwlock_wlock: Fix typo 2019-10-29 18:15:26 +03:00
Rajko Stojadinovic b49b4c8096 Rewrite the way LLVM JIT does memory allocation to allow use of more than one contiguous segment (#6771) 2019-10-29 01:01:07 +03:00
Eladash 42fc698186 rsx: Enable primitive restart index only when needed (#6889)
* rsx: Enable primitive restart index only when needed

* rsx: Use if with initializer in read_put()
2019-10-28 23:16:27 +03:00
Nekotekina 83cf6e6fa3 Qt: fix crash on startup in game_list_frame::Refresh
QList isn't supposed to be modified concurrently.
Use lf_queue to collect game_info data.
2019-10-28 22:59:56 +03:00
Nick Renieris 0b3939a58a msbuild: Add 7zlib to rpcs3 project dependencies 2019-10-28 21:00:25 +03:00
Max 8c3d2363bf [msbuild] 7zlib msbuild changes
- move project file to 3rdparty for consistency
 - add common_default props
 - fixes 7zlib.vcxproj.FileListAbsolute.txt being detected as change
2019-10-28 18:59:16 +03:00
Hula Hoop 200ec5c4b3 Super-important PR (#6900)
* Fix capitalisation errors and typos

* Oops
2019-10-28 17:44:44 +03:00
kd-11 479d92d075 vk: Fix uninitialized (and wrong) variable access 2019-10-28 15:20:45 +03:00
kd-11 b0708367c2 vk: Round lod bias to the nearest 0.5 to lower number of permutations when nearest mipmap sampling is used
- The lambda values will be rounded to the nearest integer anyway
2019-10-28 15:20:45 +03:00
kd-11 3e8dfede1c vk: Modify sampler cache to uniquely identify all the input parameters
- Avoids iteration when variable mipmap counts or lod bias parameters change
2019-10-28 15:20:45 +03:00
kd-11 ad2add9574 rsx:: Use fcmp correctly 2019-10-28 15:20:45 +03:00
kd-11 d04241ad25 rsx: Allow compressed textures to be unaligned in size
- Align based on row length but let the texture itself be of arbitrary dimensions
2019-10-28 15:20:45 +03:00
Emmanuel Gil Peyrot 69e9ee26f6 rsx: Make input_is_swizzled a template parameter
This lowers the relative cost of this function from ~2.25% to ~1.80% on
gcc 9 which I found quite surprising, some of it probably gets inlined
better in the callers, but I haven’t been able to isolate which parts.
2019-10-28 13:28:51 +03:00
Nekotekina 46d692d5a6 Fix long path support in fs::statfs on Windows 2019-10-27 19:17:27 +03:00
Nekotekina 839e088ae5 Fix long path support on Windows for UNC path
Such as network locations.
2019-10-27 18:53:40 +03:00
Eladash 65e47490c4 cellSysCacheClear/Mount() improved
Clear() error checking simplified a bit
Mount() now clears cache if ID was changed from last or NULL specified.

Implemented vfs::host::remove_all():
Clear() now uses vfs::host::remove_all() to match behaviour on Windows with ps3
2019-10-27 15:56:16 +03:00
Nekotekina 83f253636a SPU LLVM: verbose patchpoints in "giga table" 2019-10-27 13:04:05 +03:00
Nekotekina 16edb5bdd0 SPU Analyser: don't compile on "analyser failed" 2019-10-27 13:04:05 +03:00
Nekotekina 5f00b32867 SPU: fixup for duplicates in recompilers
Fixup for the edge case.
2019-10-27 13:04:05 +03:00
Nekotekina 37dbbce194 SPU LLVM: Enable "Giga table" in Mega mode
Was previously only enabled in Giga mode.
Also fill the table with branch patchpoints.
2019-10-27 13:04:05 +03:00
Megamouse 6fe32887b2 Qt: disable progress_dialog resize 2019-10-27 08:02:45 +01:00
Megamouse 200162cf2a Qt: connect compat error handling to error signal 2019-10-27 08:02:45 +01:00
Megamouse 59747fd708 Qt: add quickstart link to SSL warnings 2019-10-27 08:02:45 +01:00
Nekotekina 8c28c4e8ec SPU: Make recompilers lock-free. 2019-10-26 23:33:18 +03:00
Nekotekina 9ac6ef6494 SPU: cleanup former OOM handling
Remove cpu_flag::jit_return.
It's obsolete now, and worked only in SPU ASMJIT anyway.
2019-10-26 21:24:12 +03:00
Nekotekina 7a32af7a57 Implement lf_bunch<>
Simple concurrent container without erase support.
2019-10-26 21:24:12 +03:00
Nekotekina 2e85b5ebaa SPU: Make JIT allocator lock-free on all platforms 2019-10-26 21:24:12 +03:00
Nekotekina 0c06456950 Simplify lf_queue<> (second attempt)
It doesn't need reinterpret_cast anymore.
2019-10-26 21:24:12 +03:00
msuih a4d8cab67e Qt version bump
Also gives a tip about the Qt5_DIR definition.
2019-10-26 21:23:38 +03:00
MSuih f3ed26e9db Small warnings cleanup (#6671)
* Ignore more warnings

These are intentional

* Signed/unsigned mismatch when comparing

* Explictly cast values

* Intentionally discard a nodiscard value

* Change ppu_tid to u32

* Do not use POSIX function name on Windows

* Qt: Use horizontalAdvance instead of width

* Change progress variables to u32
2019-10-25 13:32:21 +03:00
Megamouse 89a2d04f96 Qt: add mutex to game list creation
should prevent random crashes when accessing maps
2019-10-25 12:10:07 +02:00
arabek 63a0b97dd1 Normalize audio when downmixing to avoid clipping (#6867)
* Normalize audio when downmixing to avoid clipping

Idea came from this topic:
https://hydrogenaud.io/index.php?topic=104214.msg855199#msg855199

Fixes very loud audio in Motorstorm (and probably other games
when playing over headphones/stereo speakers with
Downmix to Stereo option enabled)
2019-10-25 12:56:47 +03:00
Oschowa 6944e180c8 rpcs3qt/settings_dialog: fix Audio Buffering setting for FAudio backend 2019-10-25 00:56:53 +03:00
Oschowa 9e753e1ffd 3rdparty: update FAudio to 19.10
and add 'ignore = dirty' to faudio submodule
2019-10-25 00:56:53 +03:00
Oschowa d8ff7c759b 3rdparty: only build FAudio with SDL >= 2.0.9 2019-10-25 00:56:53 +03:00
Oschowa 06433d614a Implement FAudio backend (#6374) 2019-10-24 22:26:29 +03:00
RipleyTom a32f979814 Conserve error value when trying to open log file 2019-10-24 22:22:15 +03:00
RipleyTom c89ad38ef1 Check if we're in the process of restarting when trying to open the log 2019-10-24 18:36:28 +02:00
Marin Baron 4599d58413 Fix debug build gcc/clang linker.
Create temporary variable to resolve undefined reference.

[ 98%] Linking CXX executable ../bin/rpcs3
/usr/bin/ld : Emu/librpcs3_emu.a(sys_vm.cpp.o) : dans la fonction « sys_vm_t::~sys_vm_t() » :
/mnt/raid/Git/rpcs3/rpcs3/Emu/Cell/lv2/sys_vm.cpp:21 : référence indéfinie vers « id_manager::id_traits<sys_vm_t, void>::invalid »
collect2: erreur: ld a retourné le statut de sortie 1
make[2]: *** [rpcs3/CMakeFiles/rpcs3.dir/build.make:426: bin/rpcs3] Error 1
2019-10-24 14:34:38 +03:00
JohnHolmesII 0842724f94 Update cellVoice.cpp
Caused warnings. Not sure what the actual intention was, this may need to be inverted.
This commit assumes that erase() returning 0 is a sign that deletion
failed, or that there was corruption. There should be a port there.
2019-10-24 14:19:52 +03:00
Emmanuel Gil Peyrot 9253f7d645 Audio: don’t build XAudio2 backend on Linux
The files already had a #ifdef _WIN32, but this avoid even trying to
compile their translation unit.

I was surprised to see XAudio2 being mentioned on Linux, this makes sure
no one else will get this surprise.
2019-10-24 14:16:29 +03:00
Megamouse a360836fef Qt: don't update the gui from QtConcurrent 2019-10-24 00:26:07 +02:00
Megamouse 56a249fd5a Qt: multi thread trophy loading 2019-10-24 00:26:07 +02:00
Megamouse 221508fa07 Qt: indent try and catch block 2019-10-24 00:26:07 +02:00
Megamouse b805238b8a Qt: multi thread game list refresh from drive 2019-10-24 00:26:07 +02:00
Megamouse 5b59c18dcc Qt: multithreaded icon resize 2019-10-24 00:26:07 +02:00
Nekotekina 6867fc5f06 Revert "Simplify lf_queue<>"
This reverts commit 050e5b4aec.
2019-10-23 21:55:31 +03:00
Megamouse d0ea71289e fix weird filters for display sleep files
For some reason kd-11 thought generated files was a good place to put them
2019-10-23 20:35:32 +02:00
Megamouse c88654bc8f Remove VS D3D12 Dependencies 2019-10-23 20:35:32 +02:00
Nekotekina ed0c25f760 Update LLVM (fixup) 2019-10-23 18:10:47 +03:00
Nekotekina ec1ea466fd atomic.hpp: fix typo 2019-10-23 16:01:14 +03:00
Nekotekina 050e5b4aec Simplify lf_queue<>
It doesn't need reinterpret_cast anymore.
2019-10-23 16:01:14 +03:00
Nekotekina 6e19881b82 Update LLVM (10) 2019-10-23 16:01:14 +03:00
kd-11 d53d7bb598 vk: Restore vega native use of FP16 in shaders
- AMD proprietary drivers should work fine
2019-10-23 12:20:06 +03:00
Emmanuel Gil Peyrot 54d95373d0 Support fullscreen properly on Wayland
The current behaviour when going fullscreen from windowed was to keep
the previous size of the swapchain, with black borders on all sides,
which looks quite ugly.

The root of this issue is that rpcs3 only checks for frame resize if
vkQueuePresent() returns VK_SUBOPTIMAL_KHR, which drivers can’t do on
Wayland, see https://gitlab.freedesktop.org/mesa/mesa/issues/1979
2019-10-23 12:19:46 +03:00
Megamouse 2cc71afdcd Remove further occurrences of DirectX 12 2019-10-23 00:03:20 +02:00
kd-11 e04b6cd7c0 rsx: Copypasta fix
- r1 is always float4 never half4. Its a full-width register unlike the
other outputs which are optionally half-width.
2019-10-23 00:50:24 +03:00
Zion f1d474f932 Bump version built on to 16.04, Update clang to 8, update Qt to 5.12.3 2019-10-22 21:52:01 +03:00
kd-11 00bc3fe658 Drop d3d12 backend 2019-10-22 21:45:14 +03:00
Emmanuel Gil Peyrot 14c63ec014 Fix misleading indent. 2019-10-22 16:11:43 +03:00
Emmanuel Gil Peyrot 9d1b7af2eb Avoid std::move() for copy elision
This could disable zero-copy optimisations, which is the opposite of
what we want.
2019-10-22 16:11:43 +03:00
Eladash 586fe11e22 Fix cellGcm HLE regression
Also correct flags.
2019-10-22 13:45:09 +03:00
Eladash 945abcc6cd rsx: Align down index array offset
* Also use improved to_be_t<> template (recetly ignoring one byte long types) for vm gsl::byte referencing, remove redundent narrow<> cast (same type)
2019-10-22 13:45:09 +03:00
kd-11 3bb70e837a vk: Silly copypasta 2019-10-22 13:44:49 +03:00
kd-11 0b2f9f0f17 rsx: Add support for delayed shader discard.
- Noticed a glitch on AMD hw and windows drivers where discard seems to affect entire 4x4 cells.
- Dead fragments (outside the primitive boundary) could have their discards trigger as they do not have proper access to variables.
- This introduces dead fragments along triangle edges, causing a diagonal line pattern across the screen that is very annoying.
2019-10-22 13:44:49 +03:00
kd-11 901942f24a rsx: Replace pointless f32[4] restriction on texture parameters.
- Use a struct instead to improve readability and remove pointless OpBitCast
2019-10-22 13:44:49 +03:00
kd-11 f7842b765f rsx: Implement packed format renormalization
- Renormalizes arbitrary N-bit values as 8-bit normalized.
- NV hardware performs integer normalization at 8 bits if the size is less than 8.
- This can cause significant arithmetic drift because the error is multiplied by a huge number when sampling.
2019-10-22 13:44:49 +03:00
Ani 407be45069 Partially revert git-version.cmake 2019-10-22 11:42:49 +01:00
RipleyTom f8e4079c41 Fix for auto updater 2019-10-22 07:43:12 +02:00
RipleyTom 9f9ac4b696 Auto-updater 2019-10-22 01:51:50 +02:00
RipleyTom 56e6dd1c46 Adds 7z sdk to 3rdparty 2019-10-22 01:51:50 +02:00
Nekotekina f7ebc55326 MSVC: disable new exception handling (FH4)
Should fix countless issues with vcruntime140_1.dll not found.
2019-10-21 23:52:20 +03:00
Nekotekina b329bb604c SPU LLVM: implemented asynchronous compilation
Implemented interpreter-based pre-recompiler.
Interpreter functions are build with SPU LLVM.
2019-10-21 19:29:34 +03:00
Eladash 29cddc30f0 rsx: Fix vblank signals flood after Emu.Resume() 2019-10-21 15:31:45 +03:00
Eladash cbecbc264e cellGcm: Fix driver_info->memory_size
based hw test
2019-10-21 15:31:45 +03:00
Eladash 5de0005f5a rsx: Report full method range on invalid methods
Also report full command on fifo desync event for the first time
2019-10-21 15:31:45 +03:00
eladash 730e9cde84 sys_rsx: Improve allocations and error checks
* allow sys_rsx_device_map to be called twice: in this case the DEVICE address retrived from the previous call returned
* Add ENOMEM checks for sys_rsx_memory_allocate and sys_rsx_context_allocate
* add EINVAL check for sys_rsx_context_allocate if memory handle is not found
* Separate sys_rsx_device_map allocation from sys_rsx_context_allocate's
* Implement sys_rsx_memory_free; used by cellGcmInit upon failure
* Added context_id checks
* Throw if sys_rsx_context_allocate was called twice.
2019-10-21 15:31:45 +03:00
Marin Baron 90aaaceba0 Debug build : Check ObjectFile before reading.
Check LLVM ObjectFile state before access to avoid llvm assertion failure.

Expected<T> must be checked before access or destruction.
Expected<T> value was in success state. (Note: Expected<T> values in success mode must still be checked prior to being destroyed).
2019-10-21 00:40:12 +03:00
Nekotekina b844cd81e8 atomic.hpp: remove <string> dependency 2019-10-20 23:06:17 +03:00
Nekotekina 2dde133e1f atomic.hpp: fix internal resource deallocation
Also remove debug printf.
2019-10-20 23:06:17 +03:00
kd-11 3c44065684 gl: Fix copypasta
- MSAA is still unimplemented in OGL
2019-10-20 21:38:40 +03:00
kd-11 f40f2c6215 vk: Fix minification filter description for NEAREST_MIPMAP_NEAREST. Just a typo.
- Also remove mipmap filter for CONVOLUTION
2019-10-20 21:38:40 +03:00
kd-11 09de3b7974 rsx: Tweak behaviour of the "Use GPU texture scaling" option
- If either source data or dest is a render target, do image operations on the GPU same as before
- If swizzle is desired, use CPU fallback
- If no scaling and no format conversion is required, use CPU fallback
- If scaling is desired and the transfer target is in local memory, use the GPU
- When doing trivial copies, use the routine in rsx_methods instead of
  duplicating code. Also has the benefit of better range checking.
2019-10-20 21:38:40 +03:00
kd-11 868547aec8 rsx: Minor improvement to fbo region invalidation
- When commiting a block as fbo, keep blit_dst data as well.
- Avoids removing (and losing data from) blit targets that just happen to share a page with a framebuffer.
2019-10-20 21:38:40 +03:00
Eladash c16319f959 PPU Precise: Fix fused float ops misaccuracy by using fma
Also cleanup add64_flags, remove redundent copy of s_ppu_itype.
2019-10-20 20:58:31 +03:00
Nekotekina 4d46e66833 atomic.hpp: improve internal data layout
Put semaphore control variable together in hashtable.
Avoid unnecessary allocation of extents.
2019-10-20 20:08:09 +03:00
Nekotekina 56b310ebac atomic.hpp: reimplement USE_FUTEX path for Linux 2019-10-20 20:05:36 +03:00
Nekotekina e0f60c5dce atomic.hpp: rewrite collision handling
Remove "fallback" code path.
Remove USE_FUTEX code path temporarily.
2019-10-20 18:29:20 +03:00
Eladash 79a3a7ce4c Add exception for sys_usbd_initialize 2019-10-20 15:28:26 +01:00
Eladash a1e7cdf1e9 Forcefully awake waiters in sys_usbd_finalize
This is a no-op from cellUsbd side but happens on lv2 side as well.
2019-10-20 15:28:26 +01:00
Eladash d90696be90 sys_usbd Tiny update 2019-10-20 15:28:26 +01:00
Eladash 6c2307fb96 cellVoice: Initial Implementation
Implement system service events, error checking and some functions.
2019-10-20 15:16:53 +01:00
Eladash fa73175754 sceNpBasicGetEvent error checks improved 2019-10-20 14:56:44 +01:00
Eladash d9a97d8674 sceNpUtil Improvements 2019-10-20 14:56:44 +01:00
kd-11 996534c559 rsx: Fixup for aspect mismatch 2019-10-20 15:25:07 +03:00
Megamouse 0ef7ad129d cellNetCtl: improve param checks 2019-10-19 19:20:18 +02:00
Megamouse e6aeb7ecb9 stub cellKey2Char 2019-10-19 19:13:33 +02:00
Nekotekina 59382e023d Compilation fix 2019-10-19 14:08:21 +03:00
Nekotekina 3304cfacde cond_variable: improve wait a bit
Mask signal bits
2019-10-19 14:02:13 +03:00
Nekotekina 2446bb5d40 init_mutex: implement init_always() 2019-10-19 14:01:34 +03:00
Nekotekina db7042b986 Restore using keyed events in waitable atomics on Windows 2019-10-18 23:08:00 +03:00
Eladash d4ba7f37b6 rsx util: Implement decode_fxp<> 2019-10-18 15:41:39 +03:00
kd-11 299b98b30a vk: Disable mipmap sampling if sampling mode is does not have a mipmap filtering mode.
- GL_LINEAR and GL_NEAREST always sample LOD0 so make vulkan behave the same way
2019-10-18 14:46:37 +03:00
kd-11 404073c74a rsx: Force-align compressed formats to 4x4 texel blocks and disable 1D compressed textures.
- The PS3 allows defining 1D compressed images but this obviously doesn't work well on desktop.
2019-10-18 14:46:37 +03:00
kd-11 eff4e95c99 rsx: Minor cache fixup for cyclic references.
- Logic was broken by mipmaps PR. Do not issue a texture barrier if a temp copy is being done.
2019-10-18 14:46:37 +03:00
kd-11 bd1bcc6be7 vk: Remove a redundant memory barrier 2019-10-18 14:46:37 +03:00
kd-11 70642484cd vk: Check for cyclic references if sampler is marked as do-not-cache.
- Usually an indication of surface/texture cache interaction.
2019-10-18 14:46:37 +03:00
kd-11 eee2237e19 rsx: Track uncached cache resources
- Uncacheable resources can be reused as soon as they're made visible to the draw call.
- Since they're likely to be reused every draw call until the shader changes, it is important to reuse as much as possible
2019-10-18 14:46:37 +03:00
kd-11 decf9cfcf6 rsx: Notify the backend to release or delete temporary surfaces after we're done with them. 2019-10-18 14:46:37 +03:00
kd-11 97ed95d21b vk: Add video memory manager to monitor VRAM usage 2019-10-18 14:46:37 +03:00
Megamouse d6664450df Input: avoid exception while no keyboard is registered 2019-10-18 09:20:51 +02:00
kd-11 1046184dd0 rsx: Fix some uninitialized variables flagged by valgrind 2019-10-18 00:32:38 +03:00
Megamouse 0bad56da67 cellScreenshot: initial implementation 2019-10-17 21:24:25 +02:00
Markus Stockhausen 2850d0c6bc avoid update of window title
only update title if user really wants to display the FPS during gameplay.
2019-10-17 21:15:14 +02:00
Megamouse 60f3ed7607 stub cellSsl 2019-10-17 20:32:33 +02:00
Nekotekina 00895862e1 Fix sys_vm_t destructor
It's not a destructor's job to free vm like this.
2019-10-17 19:57:29 +03:00
kd-11 5af8a9fbbc rsx: Fix decoding of some fixed point texture parameters
- Checked envydocs and found the correct format as fixed-point 4.8 with optional sign bit
2019-10-17 18:18:00 +03:00
kd-11 a936e43ff6 rsx: Fixup for slice gathering for structures with multiple mipmap levels
- TODO: Proper multi-level assembly for non-2D structures
2019-10-17 18:18:00 +03:00
kd-11 e47b4ffb8f rsx: Fix rsx capture crash.
- Pixel coordinates are top-left not bottom-right
- Solves out of bounds access
2019-10-17 18:18:00 +03:00
kd-11 e166dbccc8 rsx: Fix visibility of blit destination targets 2019-10-17 18:18:00 +03:00
kd-11 0c35595ce2 rsx: Remove the alpha-to-coverage hack that was added to hide the missing mipmaps in games
- Moves to a purely stochastic function using dithering to simlulate coverage
2019-10-17 18:18:00 +03:00
kd-11 f0ed0285f3 rsx: Implement range-based subresource descriptor cache
- The previous address-based approach was pretty awful when it comes to invalidating
2019-10-17 18:18:00 +03:00
kd-11 fbb9ed4e25 rsx: Add explicit range to cached subresource descriptors 2019-10-17 18:18:00 +03:00
kd-11 c9e3a321b2 rsx: Fixup for surface cache scanning
- Fix regression when gathering cubemaps
2019-10-17 18:18:00 +03:00
kd-11 1ac976771c rsx: Add some texture search options for the cache
- Potentially optimizes texture cache searching using explicit options
2019-10-17 18:18:00 +03:00
kd-11 840b52fe80 rsx: Implement mipmap gathering from texture cache 2019-10-17 18:18:00 +03:00
kd-11 d6d8766f8d rsx: Refactoring
- Move some helper routines out of the cache core
- Prep for multi-layered image search
2019-10-17 18:18:00 +03:00
kd-11 cb362b4085 rsx: Runtime check on RTT cast 2019-10-17 02:30:03 +03:00
kd-11 5c7bbb3354 vk: Fixup
- Removes incorrect line writing stencil flags to a regular texture.
2019-10-17 02:30:03 +03:00
kd-11 d29b6cdb59 vk: Proper workaround for VEGA float16_t bugs 2019-10-16 22:40:50 +03:00
kd-11 a6e143254a vk: Add workaround for broken format conversion in older GeForce cards 2019-10-16 22:40:50 +03:00
kd-11 4f088a102c vk: Add kepler and maxwell tables 2019-10-16 22:40:50 +03:00
eladash 95752607ea sys_spu: Don't reset snr config at group_start()
Also first check for EINVAL in sys_spu_thread_set_spu_cfg
2019-10-16 21:11:29 +03:00
Eladash 6dd96d3db9 Fixes for cellUserInfoGetList
- Don't allow all args to be nullptr at once.
- Fill listBuf with zeroes for unwritten entries
- Fix userId set in listBuf

Similarly to what the firmware does
2019-10-16 21:11:29 +03:00
Eladash 47c3d945aa spu/ppu-diasm: Fixup for #6772, #6722 2019-10-16 21:11:29 +03:00
Nekotekina 0fe46934de Fix "SPU Interpreter (fast)"
Regression after #6770
2019-10-16 14:20:35 +03:00
Eladash 4ed2c7b594 error checking bugfix for sceNpTrophyCreateContext
options = 1 is allowed.
2019-10-16 00:50:58 +03:00
Eladash b03c2fc856 sys_cond_wait: Fix mutex acquire when signaling thread didnt pass ownership immediatly 2019-10-16 00:50:58 +03:00
Eladash ab02c56725 sceNpTrophyRegisterContext verbose logging in TROPHY_ERROR_ILLEGAL_UPDATE error
Also use safer checking when creating trophy ctxt dir
2019-10-16 00:50:58 +03:00
Eladash 80b4ac23e5 spu disasm: Show interrupts status modifiers on indirect branches
That's how the assembler represents it as well.
2019-10-16 00:50:58 +03:00
Nekotekina 7ca76ae5a8 Workaround fs::create_path for drive roots
Should fix the regression from #6763
2019-10-15 18:57:16 +03:00
Nekotekina eafbc77c0d SPU LLVM: Always use linux-gnu target triple (affects Windows)
Unify internal code generation to make better use of GHC calling convention.
Ideally, it would just work on Windows as well, but some random bug appeared.
This bug was causing freezes on SPU LLVM compilation.
This commit desperately attempts to workaround it.
2019-10-15 18:04:50 +03:00
Nekotekina c69fe0f664 SPU Profiler preview
Add option "SPU Profiler" (disabled by default).
Works only with SPU recompilers.
Results are flushed on pausing.
2019-10-15 17:42:57 +03:00
Malcolm Jestadt 1a9e06d3c6 Linux: Change default Sleep Timers accuracy to host
- This doesn't change existing configs
- Also sets the host_min_quantum to the true value
- Restores lost TODO: comment
2019-10-15 00:32:31 +03:00
plappermaul 2171ffdab2 minor optimization for FIFO_control::read_put() (#6768) 2019-10-14 21:26:31 +03:00
Eladash 7e08fff91d ppu disasm: Improve CR bits manipulation instructions formatting 2019-10-14 20:44:23 +03:00
Eladash 3910b2465e Fix unknown BCLR forms formatting 2019-10-14 20:44:23 +03:00
Eladash 75ee668922 Log fs::g_tls_error on ACCESS_ERROR errors 2019-10-14 20:44:23 +03:00
Nekotekina 076a80d71e Random space fixes 2019-10-14 19:42:35 +03:00
Nekotekina 6b1df1ce31 fs: add long path support on Windows
Extend path limit with \\?\ prefix.
2019-10-14 19:42:13 +03:00
Ani 4d0291ed71 Hotfix for #6573
Fix path for rpcs3_version.cpp
2019-10-14 15:52:24 +01:00
Ani 85b8583208 Hotfix for #6573 2019-10-14 14:26:13 +01:00
Ani f8153fca58 Unify versioning
Use the same version value across all scripts instead of program side version and repository side version separatedly
2019-10-14 13:57:36 +01:00
Ani 6a0ce010eb Update git-version generation 2019-10-14 13:57:36 +01:00
Eladash 1e303e9f97 ppu disasm: Implement BCLR's, BCCTR's and BC's extended mnemonics
Also:
BCCTR cr bit is now shown as cr(bit/4)[bit % 4]
BLRL is now shown properly
BDN?Z[TF](LR?)L?A? branches are now implemented.
2019-10-14 13:54:56 +01:00
Eladash 5978b1f28f ppu disasm: Fix AA BC formatting 2019-10-14 13:54:56 +01:00
Nekotekina c4ccac9a9c [TSX] Fix XABORT handling
Break repeat loop after explicit transaction abort.
May have performance consequences.
2019-10-13 19:35:12 +03:00
Eladash 1d07c40aa5 Fix state reset in cellMic, cellAudio, cellCamera (#6761) 2019-10-13 19:26:11 +03:00
kd-11 42aa4c5000 gl: Vendor-specific tuning 2019-10-13 19:00:05 +03:00
kd-11 776fa54d22 gl: Fix missing case 2019-10-13 19:00:05 +03:00
kd-11 27f48fbc06 gl: Rewrite image transfer operations to support image subregions
- Working exclusively with full sized images is very expensive
2019-10-13 19:00:05 +03:00
kd-11 d9a9766e41 gl: Refactoring and fallback support for compute acceleration 2019-10-13 19:00:05 +03:00
kd-11 b39bfa02a6 gl: Windows bringup 2019-10-13 19:00:05 +03:00
kd-11 105d4b51e6 gl: Use compute shaders for typeless texture decode 2019-10-13 19:00:05 +03:00
kd-11 7a6e2e716f gl: Add a framework for compute shaders 2019-10-13 19:00:05 +03:00
Alex James 9bcd7b534b CMake: Generate compilation database 2019-10-13 00:02:33 +03:00
Nekotekina ac712bf4e3 lf_queue: remove explicit notify(), notify in push() instead
Should be cheap enough now if there are no waiters.
2019-10-12 22:45:17 +03:00
Nekotekina 5624b001ae lf_queue: add range-for support (endless loop with waiting) 2019-10-12 22:37:52 +03:00
Nekotekina 16dd72b3e3 Replace some uses of Emu.IsStopped()
Poll thread state instead.
2019-10-12 21:12:47 +03:00
Nekotekina 9ae08946ac g_fxo: destroy objects in reverse creation order
Maintain creation order internally.
2019-10-12 20:58:36 +03:00
Nekotekina 6a443c0fbd Deprecate thread_ctrl::spawn 2019-10-12 19:14:50 +03:00
Nekotekina 8e21f4d5d4 Remove g_thread_count
Unnecessary global which is bad to reason about.
Possibly fix a but in SPRX loader.
2019-10-12 16:45:41 +03:00
Nekotekina 49e96b39dd [SPU, TSX] Fix reservation corruption in PUTLLC
Change reservation locking logic.
2019-10-12 15:41:24 +03:00
Alex James 3ad743ecaa Add option for preventing display sleep (#5783)
Adds support for preventing the display from sleeping while a game is
running. Supports Windows, Linux (with the org.freedesktop.ScreenSaver
D-Bus service), and macOS.
2019-10-12 15:40:47 +03:00
Megamouse 07022fd3b6 stub cellVoice 2019-10-12 10:16:01 +02:00
Eladash 0172e097f7 Fix cellCameraInit (camera_handler::null) 2019-10-11 21:55:10 +02:00
Eladash 5cc12ef22e cellCamera: Log error code 2019-10-11 21:55:10 +02:00
Markus Stockhausen 4d99169d51 Patch v2 for vkCreateInstance()
as requested
2019-10-11 21:16:36 +03:00
Markus Stockhausen 8adcb8046b Patch for vkCreateInstance()
patch as requested
2019-10-11 21:16:36 +03:00
Markus Stockhausen f5817cb430 Error handling for vkCreateInstance()
Cry in log if initialization failed.
2019-10-11 21:16:36 +03:00
Eladash 397007cf8b rsx: Fix FIFO_DRAW_BARRIER substituation 2019-10-11 12:34:53 +03:00
Eladash 9242f16560 rsx: Improve FIFO recovery from flip 2019-10-10 19:34:23 +03:00
Eladash 06017cb14e rsx: Recover from invalid writes to CELL_GCM_NV4097_SET_INDEX_ARRAY_DMA
Also: Trigger a FIFO recovery when encountering an invalid method.
2019-10-10 19:34:23 +03:00
Eladash 2eaf5df60b rsx: Register some more methods 2019-10-10 19:34:23 +03:00
msuih e9ea226e30 Validate firmware before installing 2019-10-09 21:21:23 +03:00
plappermaul 925f2ce02f Use Linux timers for sleeps up to 1ms (#6697)
* Use Linux timers for sleeps up to 1ms (v3)
The current sleep timer implementation basically offers two variants. Either
wait the specified time exactly with a condition variable (as host) or use a
combination of it with a thread yielding busy loop afterwards (usleep timer).

While the second one is very precise it consumes CPU loops for each wait call
below 50us. Games like Bomberman Ultra spam 30us waits and the emulator hogs
low power CPUs. Switching to host mode reduces CPU consumption but gives a
~50us penalty for each wait call. Thus extending all sleeps by a factor of
more than two.

The following bugfix tries to improve the system timer for Linux by using
Linux native timers for small wait calls below 1ms. This has two effects.

- Host wait setting has much less wait overhead
- usleep wait setting produces lower CPU overhead
2019-10-09 20:03:34 +03:00
Nekotekina 6b1e1e4020 GDB Server: parse 'GDB Server' config entry
Add support for unix sockets.
2019-10-09 16:41:01 +03:00
Nekotekina 91dcc3f60c GDB Server: fix selecting ppu_thread 2019-10-09 03:17:10 +03:00
Nekotekina 23bbeb7f8f GDB Server: fix ppu_thread access cast 2019-10-09 03:12:45 +03:00
Nekotekina ad5afc074e GDB Server: format cleanup 2019-10-09 03:06:59 +03:00
Nekotekina 8f604ddded Fix log format misuse
Harden log channel methods against non-constant string.
2019-10-09 02:14:52 +03:00
Nekotekina a29d4150df GDB Server: fix and cleanup
Move source files to Emu/GDB.cpp, GDB.h
Remove "WITH_GDB" option, enable GDB Server by default.
Change class name to gdb_thread.
Alias for external access gdb_server.
Change config option name to "GDB Server"
Bind on 127.0.0.1 by default.
2019-10-09 00:11:44 +03:00
Nekotekina 9d4de51cb6 Use g_fxo for network thread
Minor refactoring
2019-10-08 23:12:54 +03:00
Eladash 0f3c5f9e5a Fix verification failure on Emu.Stop() at sys_cond_wait 2019-10-08 23:12:34 +03:00
Eladash af417c93d0 cellGemUpdateStart/Finish error checking improved
* camera_frame = NULL is now checked for CELL_GEM_NO_VIDEO (applied both on start and finish)
* camera_frame = NULL on Start() still starts the update
* Implemented NOT_FINISHED/STARTED

All of those were reversed and hw tested.
2019-10-08 23:12:34 +03:00
Megamouse 278050f8d0 add cellPadFilter functions 2019-10-08 08:21:30 +02:00
Megamouse 29c77132cb add error code to cellPadInfoPressMode and cellPadInfoSensorMode 2019-10-08 08:21:30 +02:00
Megamouse f8c6cc2476 cellMic: add some params
makes me wonder if dev_num should always be s32
2019-10-08 08:21:30 +02:00
Eladash 36de3d4f4c sys_gpio Cleanup 2019-10-08 02:52:33 +03:00
Eladash c2278fb879 spu: Mask SRR0 at write 2019-10-08 02:52:33 +03:00
Megamouse 3455d76970 cellPadGetData: clear buttons if input is ignored but not intercepted.
needed in Hotline Miami and NinJa Gaiden Sigma, fixes Gran Turismo 6 regression
2019-10-07 19:25:50 +02:00
Megamouse d402507be2 cellOskDialog: add some param checks 2019-10-07 19:25:50 +02:00
Nekotekina aa659f3c36 Fix minor warning 2019-10-06 20:40:53 +03:00
Nekotekina cf13dee1f3 Prioritize vip_lock in shared_mutex
Should fix some deadlock issues.
2019-10-06 16:03:38 +03:00
Nekotekina 239f53568c SPU: optimize function dispatch in trampolines
Add a top-level hashtable
2019-10-06 16:03:38 +03:00
Nekotekina 8031180373 Add dummy alert param to thread_ctrl::wait API 2019-10-06 16:03:10 +03:00
Nekotekina 86fb2993ad Minor fixup for se_t operators &= |= ^= 2019-10-06 13:33:50 +03:00
Silent f5d8110ead Create emulator directories after changing dev_*** directories (#6707)
* Make pkg_install create path to the game recursively

* Create emulator directories after changing dev_*** dir paths
2019-10-05 16:23:12 +03:00
Eladash 9e929a25d4 rsx: Default initailize zcull reports padding 2019-10-05 15:00:50 +03:00
Rajko Stojadinovic 297e3c5996 Make windows_file::stat return the same mtime as fs::stat 2019-10-05 14:25:34 +03:00
kd-11 305a5bd717 typo fix 2019-10-05 12:01:46 +03:00
Megamouse 04a6b065b4 Qt: use Qt standard button layout in vfs_dialog 2019-10-04 23:06:05 +02:00
Megamouse f97ed7738d Qt: use Qt standard button layout in settings_dialog 2019-10-04 23:06:05 +02:00
Megamouse 91f53a366b Qt: fix high dpi setting 2019-10-04 23:05:07 +02:00
Megamouse f7ec679242 mmjoy: fix connection update spam 2019-10-04 20:22:32 +02:00
Megamouse df5af50866 evdev: add missing override
removes a warning
2019-10-04 20:22:32 +02:00
kd-11 4a19a2dd24 rsx: Explicity describe transfer regions for both source and destination blocks 2019-10-04 18:10:46 +03:00
Eladash 08e674aa28 sys_cond fixes (#6584)
* sys_cond fixes

sys_cond_wait is now signaled atomically (regression fix)
Fix a corner case with sys_cond_wait and sys_cond_destroy EBSUY check (waiter count was inaccurate if thread is not the owner of mutex)
Add not about EBUSY corner case (TODO)

* Fix inconcistency in sys_cond_destroy and ETIMEDOUT

 .. event at sys_cond_wait regarding waiters count.

Now waiters count is properly connected to the mutex owner actions after ETIMEDOUT in sys_cond_wait.
2019-10-03 23:05:34 +03:00
Eladash 286d62d09c sceNp2Init: use u32 in poolsize 2019-10-02 01:52:42 +01:00
Nekotekina f83385926b Implement optimized operators &|^ for se_t 2019-09-30 22:32:39 +03:00
Nekotekina e56ffc1e69 Implement optimized operator~ for se_t<> 2019-09-30 22:26:34 +03:00
Nekotekina 5f29840348 Fix enum vs integer comparison in se_t::operator==
Need to convert left part to its underlying type.
2019-09-30 22:26:34 +03:00
Nekotekina 0f567abdd8 Restore experimental optimized operators &= |= ^= for se_t
They were removed approximately 3 years ago due to their rarity.
2019-09-30 22:26:34 +03:00
kd-11 7aed9c3f13 gl: Add missing input declarations for 2-sided lighting 2019-09-30 21:52:43 +03:00
kd-11 88229f4716 gl: Remember to unbind attachments from active framebuffer after clear
- If a stale reference is left lying around (e.g the texture bound to
depth has been deleted and we attach a color image) no operations
actually take place. glCheckFramebufferStatus also does not catch this
problem.
2019-09-30 21:52:43 +03:00
Eladash 0b2fa6ffdc rsx: Flush FIFO GET before smeaphore_acquire 2019-09-30 17:30:15 +03:00
Eladash 70b4ae6bd6 rsx: Optimize FIFO PUT masking 2019-09-30 17:30:15 +03:00
kd-11 bcf8799079 rsx: Fix missing point size export
- Sometimes program-point-size is enabled, but the vs does not actually
write to the point size register. In this case, pass the incoming point
size along instead of the default register init.
2019-09-30 01:40:04 +03:00
Megamouse 19e0d099b6 config: dynamically show_fps_in_title 2019-09-29 21:00:17 +02:00
Eladash 319fc8c55d rsx: Mask FIFO PUT on rsx execution 2019-09-29 13:05:24 +03:00
Eladash 822287b418 rsx: Avoid unsigned/signed mismatch with fifo ret addr 2019-09-29 13:05:24 +03:00
kd-11 8cfd3b56d6 vk: Increase wait timeout in case of problematic GPU loads causing heavy stutter
- When compiling LLVM objects, it is possible to starve the driver thread and cause the timeouts to trigger
- Observed in RE6 when using SPU LLVM since the game generates a very large number of objects "infinitely"
2019-09-29 11:39:22 +03:00
kd-11 ef5b56bc48 rsx: Align width properly when normalizing to avoid fractional results being lowered to 0 2019-09-29 11:39:22 +03:00
kd-11 69c090b14a vk: Check frame descriptors before rendering in case of a flip request between begin() and end()
- There is no reason to delay async flip requests since most of the work can be handled during rendering anyway
2019-09-29 11:39:22 +03:00
kd-11 1464069476 rsx: Restructure deferred flip queue handling
- Allows frameskipping to occur naturally if RSX thread is bombarded with flip requests but just jumping to the last one if possible
- See request_emu_flip() for async frame submission and implicit skipping
- Also allows display queue to fill faster than the flip thread can drain the queue
2019-09-28 21:13:56 +03:00
Eladash b9b879c3d3 Bugfix for sceNpUtilCmpNpId
Fix checking bounds, use memcmp
Fix result store on sceNpUtilCmpNpIdInOrder
2019-09-28 16:02:55 +03:00
Nekotekina 046f8510fa Restore experimental optimized operators == != for se_t<> 2019-09-28 15:39:50 +03:00
Nekotekina 21885264f7 Improve to_se conversion template
Add sizeof check instead of filtering out 1-byte types.
2019-09-28 15:39:50 +03:00
Nekotekina bd1a24b894 Tidy endianness support (se_t) implementation
Move se_t and se_storage to util/endian.hpp
Use single template instead of two specializations.
Add minor optimization for MSVC.
Remove v128 dependency.
Try to enable intrinsics for unaligned data.
Fix minor bug in u16/u32/u64 specializations.
2019-09-28 15:39:50 +03:00
Nekotekina c7c12941bc Fix wrong comments in CPUThread.h
It's not an upcast.
2019-09-28 15:39:50 +03:00
kd-11 2275259bf5 rsx: Properly scale overlay passes to match drawable area 2019-09-28 13:24:14 +03:00
kd-11 28534e8833 gl: Remove a debug print 2019-09-28 13:24:14 +03:00
Eladash feabe71183 SPU: Rewrite BGX 2019-09-28 03:42:22 +03:00
Eladash e315b39822 SPU: Rewrite CGX 2019-09-28 03:42:22 +03:00
Eladash d0503dc937 SCE_NP_ERROR_OFFLINE logging disabled 2019-09-28 03:16:01 +03:00
Eladash 7f725e0655 sys_rsx: Format special sys_rsx invalid param error, warning fixes 2019-09-28 03:16:01 +03:00
kd-11 e53e98749f rsx: Add missing initialization 2019-09-27 21:07:56 +03:00
msuih 6979b96225 Log game version 2019-09-27 20:01:55 +03:00
Nekotekina 57f2189b1a cellSaveData: fix VFS escape usage 2019-09-27 19:04:16 +03:00
Nekotekina 465b16e786 Finally remove fxm from IdManager 2019-09-26 23:26:36 +03:00
Nekotekina c065a21b74 cellMsgDialog: use g_fxo 2019-09-26 23:26:36 +03:00
Nekotekina 3c72069ae6 cellOskDialog: use g_fxo 2019-09-26 23:26:36 +03:00
Nekotekina 50fc5dfde5 Use g_fxo for rsx::rsx_replay_thread 2019-09-26 23:26:36 +03:00
Nekotekina a6edcca6e6 Use g_fxo for spu_cache 2019-09-26 23:26:36 +03:00
Nekotekina 5f9c5e8765 Use g_fxo for rsx::thread 2019-09-26 23:26:36 +03:00
Nekotekina b48cdc2260 Use g_fxo for global ppu_module instance
Also fix autonomous PRX/SPU loading sequence.
2019-09-26 23:26:36 +03:00
Nekotekina 240b4a8bd8 cellGameData: use g_fxo for content_permission 2019-09-26 23:26:36 +03:00
kd-11 ee0633f43a vk: Add turing workaround
- Turing crashes if using the depth->color transfer hack
2019-09-26 20:12:25 +03:00
kd-11 acc986be3f vk: Add chip family detection 2019-09-26 20:12:25 +03:00
msuih 5ac631e03b Fix debug build
Seems to be a compiler bug
2019-09-25 23:20:52 +03:00
Megamouse 839fe786a2 input: fix DS3 button presses (forgot the non pressure sensitive ones) 2019-09-25 21:13:55 +02:00
Nekotekina ccf9543b44 Improve vfs::host::unlink on Windows (for sys_fs_rmdir)
Possibly fixes sys_fs_rmdir and other cases of directory removal.
Make sure the directory with deleted files always becomes empty.
For this purpose, temp files are moved to the root of the device.
2019-09-25 18:47:38 +03:00
Nekotekina cd843bda6e cellSaveData: always commit changes on error in funcFile loop
Make sure the changes which are already done are written.
2019-09-25 18:47:38 +03:00
Nekotekina f841b47b6b cellSaveData: add auto maintenance routine in Emu.Init()
This routine:
1) Removes junk backup directories
2) Fixes interrupted save data process in edge case
This case can happen if emu terminates between two atomic renames.

Also use directory renaming technique for delete op.
Also rewrite recreate operation to be part of atomic process.
2019-09-25 18:47:38 +03:00
Megamouse 297016aba3 fix some new warnings 2019-09-25 07:33:32 +02:00
Megamouse 41050c4364 input: fix missing override in keyboard pad handler 2019-09-25 00:30:23 +02:00
RipleyTom e0d501124f Only free the list if libusb_get_device_list succeeded 2019-09-25 01:07:04 +03:00
Megamouse 4594148409 Input: move some pad handler logic to the parent class 2019-09-24 21:09:24 +02:00
Nekotekina 8f47f9517a Fix fs::create_path
Don't try to create already existing parents
2019-09-24 16:18:28 +03:00
RipleyTom 4de301a961 Implement sys_usbd_event_port_send 2019-09-24 09:18:33 +02:00
JohnHolmesII c62a667fb1 Fix null pointer check (clang warnings) 2019-09-24 09:11:50 +02:00
Nekotekina aa8a7b3e8a Make errors in Emu::Init verbose and don't crash
Allow TTY.log to be disabled if file access failed
Add fs::error::isdir
2019-09-24 05:01:00 +03:00
Nekotekina 9710473a2e atomic.hpp: use native semaphores on Windows
Windows: drop keyed events
Linux: keep using native futex
Implement unused POSIX semaphore path
Implement fallback semaphore with pure std (OSX, BSD, etc)
2019-09-24 05:01:00 +03:00
Nekotekina 95c939f014 atomic.hpp: fix pointer mask 2019-09-24 05:01:00 +03:00
Nekotekina 74471e8ea3 atomic.hpp: fixup for atomic waiting 2019-09-24 05:01:00 +03:00
Nekotekina feee3838eb Revert "Revert "Remove shared_cond and simplify reservation waiting""
This reverts commit b70c08a2e8.
2019-09-24 05:01:00 +03:00
Jan Beich 5ec35c7daa rsx: unbreak build with Clang 9
ld: error: rpcs3/CMakeFiles/rpcs3.dir/main_application.cpp.o: unable to find library from dependent library specifier: opengl32.lib
ld: error: rpcs3/Emu/librpcs3_emu.a(GLGSRender.cpp.o): unable to find library from dependent library specifier: opengl32.lib
ld: error: rpcs3/Emu/librpcs3_emu.a(GLRenderTargets.cpp.o): unable to find library from dependent library specifier: opengl32.lib
ld: error: rpcs3/Emu/librpcs3_emu.a(GLVertexBuffers.cpp.o): unable to find library from dependent library specifier: opengl32.lib
2019-09-24 01:00:45 +03:00
Nekotekina 40fcd457bf cellSaveData fixup 2019-09-23 03:15:41 +03:00
Eladash 2494091353 cellSaveData: Log param error code (#6587) 2019-09-23 03:07:23 +03:00
Eladash 718491c56d ppu disasm: Improve BC formatting 2019-09-23 02:35:56 +03:00
Eladash 2c47e39f08 ppu disasm: Improve disassembly of BCTR and LWSYNC 2019-09-23 02:35:56 +03:00
Silent a44b1018b5 Add SCP extension support to XInput pad handler (#6524)
* Add SCP extension support to XInput pad handler

* Add SCP mention in xinput handler description tooltip
2019-09-22 11:22:42 +02:00
Megamouse 88388f1efc sceNpSns: add more logging 2019-09-22 10:16:55 +02:00
Megamouse 2dc71cfe51 sceNpTrophy: add errors from RE 2019-09-22 10:16:55 +02:00
Eladash 399ea2edb2 dont escape from check_state() before ensuring signal is resetted. 2019-09-21 19:55:10 +03:00
Nekotekina 3121fecc8f cond_variable: fix warning in cond.h for MSVC 2019-09-21 03:59:43 +03:00
Nekotekina e7d67f9732 Fix unwanted statichle_handler init 2019-09-21 03:59:43 +03:00
Nekotekina a797b350a0 Improve strcpy_trunc
Zero all remaining array
2019-09-21 03:59:43 +03:00
Megamouse caef52e3b3 Input: don't intercept pads after system dialogs, but still ignore input 2019-09-20 22:12:40 +02:00
Megamouse 571bb914f5 cellPad: fix out of bounds warning 2019-09-20 22:12:40 +02:00
Megamouse 8a63ac033e Input: Ignore unused reserved button (key code 0 was used twice) 2019-09-20 22:12:40 +02:00
Megamouse 7193d407b9 Input: Remove unused flush member 2019-09-20 22:12:40 +02:00
kd-11 1a892c6b1b rsx: Avoid recursion in flip handler 2019-09-20 15:08:41 +03:00
RipleyTom 7818d0767b Forces numeric locale to default 2019-09-20 11:00:05 +02:00
Megamouse aa7eb1536a overlays: fix enter button assignment in osk 2019-09-20 10:53:09 +02:00
kd-11 e0005ec347 rsx: Refactoring and improvement
- Separate displayed statistics from actual backend statistics.
  Allows asynchronous flipping to work correctly as it just uses display stats.
  The real stats are used by the frame scope marker to determine behavior like engaging the FIFO optimizer or skipping draw calls correctly.
2019-09-19 23:10:09 +03:00
kd-11 2c76f47eec rsx: Restructure flip code and frame scoping
- Add an explicit frame scope marker tied in with the queue_prepare command
  Since queue_prepare is emitted at the end of a frame, it can be used as end-of-frame in games that emit this
- If this command is not emitted, fifo flatenner and frameskip will not work
2019-09-19 23:10:09 +03:00
Eladash c50119aefa Fix sceNp2Init 2019-09-19 08:26:29 +02:00
Nekotekina 538a5cc9d8 Silence skylander message 2019-09-19 02:16:14 +03:00
Nekotekina 4b19e55fcc Use g_fxo for cellKb, cellMouse
Use init_mutex
2019-09-19 02:16:14 +03:00
Nekotekina 17d764409e Add init_mutex.hpp
Pseudo-mutex to protect initialization and finalization
2019-09-19 02:16:14 +03:00
Nekotekina 4bba1e3337 Use g_fxo for pad_handler 2019-09-19 02:16:14 +03:00
Nekotekina 3092914527 typeindices.hpp: add typeindex() helper 2019-09-19 02:16:14 +03:00
Nekotekina c7bf6421f4 typeindices.hpp: rename typeinfo_v to typelist_v 2019-09-19 02:16:14 +03:00
Nekotekina be61499790 typeindices.hpp: fix constexpr constructor
Serious bug...
2019-09-19 02:16:14 +03:00
Nekotekina 8054735cab Fix manual_fixed_typemap::init result (g_fxo)
Return uncasted pointer to created type
2019-09-19 02:16:14 +03:00
Nekotekina 353a7ff8e6 cellCamera: use g_fxo 2019-09-19 02:16:14 +03:00
Nekotekina a4951ec407 Use g_fxo for global lv2_memory_container 2019-09-18 21:24:04 +03:00
Nekotekina 20cb19618d Fix vm::reserve_map NRVO 2019-09-18 21:24:04 +03:00
RipleyTom 7e2bb0fd1f Adds initialization checks back in sys_usbd 2019-09-18 21:19:40 +03:00
kd-11 bd4d86f87a vk: Properly test MSAA sample mask when switching between states inside a RSX renderpass.
- Before, these changes would be lost if the same RTT config was used with varying mask setups
2019-09-18 15:42:59 +03:00
Nekotekina c900fb0d77 Delete g_idm
Needs to be reimplemented
2019-09-17 15:55:58 +03:00
Nekotekina 3eba0f4523 sys_usbd: use g_fxo 2019-09-17 15:23:40 +03:00
Nekotekina 7e2edc2232 sys_usbd: rearrange includes 2019-09-17 15:10:58 +03:00
Nekotekina c2fe2ac4a4 cellMic: use g_fxo 2019-09-17 02:46:34 +03:00
Nekotekina 36a528067c cellAudio: use g_fxo 2019-09-17 02:46:34 +03:00
Nekotekina 75941e62a8 named_thread: implement default thread name support 2019-09-17 02:46:34 +03:00
Nekotekina 76314520ef cellAudio: use vm::gvar to allocate working memory 2019-09-17 02:46:34 +03:00
Nekotekina 584174d371 vm::gvar: add array support 2019-09-17 02:46:34 +03:00
RipleyTom 2b8890b193 CellPadLdd functions implementation 2019-09-17 01:17:48 +03:00
RipleyTom e98c7f4e1a sys_usbd implementation 2019-09-17 01:17:48 +03:00
RipleyTom 2f884de885 Adds md5 hash to Crypto implementations 2019-09-17 01:17:48 +03:00
Windy Boy d782abb3fd TSX instruction tooltip typo
Should be "CPU" instead of "cpu"
2019-09-16 21:08:20 +02:00
youwereeatenbyalid cea975f6a6 Add a "stack mode" to the TTY. (#6542)
* Added stack mode to TTY

* fix declarations

* Fixed spacing, removed namechange to preserve user settings, removed debug string

* removed rename of "stack"

* Add menu separator and remove whitespace
2019-09-16 20:38:30 +02:00
RipleyTom 794a1fef1d Report filesize in bytes 2019-09-16 04:00:49 +01:00
msuih 8f965ccd48 Pad settings dialog improvements 2019-09-15 22:48:39 +02:00
RipleyTom fb8e70d7f6 Update CMake version in BUILDING.md 2019-09-14 23:46:12 +01:00
Nekotekina b70c08a2e8 Revert "Remove shared_cond and simplify reservation waiting"
This reverts commit 0a96497e13.
2019-09-14 00:02:48 +03:00
Nekotekina c9170bbb88 Fix lf_queue::wait
Remove incorrect loop
2019-09-13 23:52:18 +03:00
Nekotekina 223319dd2e Abort named_thread in cpu_thread::stop_all 2019-09-13 15:53:38 +03:00
Nekotekina e982871ab5 atomic.hpp: increase hashtable capacity
Double size and ignore 2 lowest bits (effectively x8)
2019-09-13 15:53:34 +03:00
Nekotekina 2fc8844315 atomic.hpp: add atomic wait mask support 2019-09-13 15:53:34 +03:00
kd-11 c59cb1bdd3 rsx: Allow only sse4.1 capable CPUs to take the accelerated index path
- Older sets lack the required min/max functionality
2019-09-13 12:28:52 +03:00
Megamouse dadfdc35f4 Qt: add --styles cli arg 2019-09-13 10:30:56 +02:00
Megamouse 2ab19efb90 Qt: Enable stylesheet cli args and add stylesheet option "None" 2019-09-13 10:30:56 +02:00
Silent 3e9ed9a17d Make "Clear shader cache" clear all caches, not just the first (#6538)
* Use QFileInfo::suffix to obtain and check for file extension when clearing caches

* Clear all shader caches for the game, not just the first
2019-09-13 09:14:52 +02:00
Megamouse 32a4c2242b sceNp: fixups 2019-09-13 06:18:49 +02:00
kd-11 52e8747b83 rsx: Workaround for exit deadlock
- Avoids games locking up when the stop button is pressed
2019-09-12 23:32:21 +03:00
kd-11 cc313b052f rsx: Improve hit testing when scanning for overlapping surfaces
- Calculate exact sizes when doing hit tests to avoid false negatives
- Defer page checking until actually require to do memory setup
- Introduce align2 helper to do non-pow2 alignments
2019-09-12 23:32:21 +03:00
kd-11 9842823a8c rsx: Check if memory actually exists when overallocating blit targets 2019-09-12 23:32:21 +03:00
kd-11 cd1345b6bb rsx: Do not use nul section if resolution scaling is active on a surface 2019-09-12 23:32:21 +03:00
kd-11 858014b718 rsx: Experiments with nul sink 2019-09-12 23:32:21 +03:00
kd-11 212ac19c11 vk: Reimplement DMA synchronization 2019-09-12 23:32:21 +03:00
kd-11 f06559412e vk: RDB fixup 2019-09-12 23:32:21 +03:00
kd-11 7fdb4976d8 rsx: Remove log spam for cond render 2019-09-12 14:08:21 +03:00
kd-11 60845daf45 rsx: Improve use of CPU vector extensions
- Allow use of intrinsics when SSSE3 and SSSE4.1 are not available in the build target environment
- Properly separate SSE4.1 code from SSSE3 code for some older proceessors without SSE4.1
2019-09-12 14:08:21 +03:00
kd-11 27af75fe71 rsx: Fixup for blit engine when moving inverted regions
- Properly calculate overlap range when sections are inverted
- Simplify transfer logic for inverted regions
2019-09-11 23:30:55 +03:00
Nekotekina c1b0934f72 Simplify thread_base::join()
Use waitable atomics
2019-09-10 19:25:39 +03:00
Nekotekina 4504ac2d12 atomic.hpp: implement collision fallback properly
Should prevent devastating effect of collisions
2019-09-10 19:25:39 +03:00
Nekotekina 0a96497e13 Remove shared_cond and simplify reservation waiting
Use atomic wait for reservations
Cleanup some obsolete code
2019-09-10 19:25:39 +03:00
Nekotekina d13ff285d1 Rewrite cond_variable to use waitable atomics
Increase max_timeout and fix max_timeout usage
2019-09-10 19:25:39 +03:00
Nekotekina 67f31c17d1 atomic.hpp: fix signal saturation logic
Make sure to notify_all at max signal count.
2019-09-10 19:25:39 +03:00
Nekotekina 92a75cfa80 Remove unique_cond
Not very useful.
2019-09-10 19:25:39 +03:00
Nekotekina a808c2aaf6 atomic.hpp: optimize internal logic
Move waiter count to highest bits to prevent false futex wakeups.
Test pointer bits properly in notify_all to avoid false wakeups.
2019-09-10 19:25:39 +03:00
Nekotekina aa99faa85d atomic.hpp: add timeout support 2019-09-10 19:25:39 +03:00
Nekotekina a45f86a4a2 Remove notifier class
Poorly implemented condition variable.
2019-09-10 19:25:39 +03:00
Nekotekina b91661ae71 Remove timeout support from lf_queue::wait
Add notify method and use atomic wait
2019-09-10 19:25:39 +03:00
Nekotekina faba366f89 named_thread: install atomic wait callback 2019-09-10 18:48:46 +03:00
Nekotekina 1668d6ee1b atomic.hpp: implement wait callback interface
Will be used to wake up threads uniformly.
2019-09-10 17:43:04 +03:00
Jan Beich 53e7f6c5a5 Thread: drop is_leaf_function after 5e3bacbd9b 2019-09-10 17:41:15 +03:00
kd-11 412c620b9d rsx: Allow sampling from shader_read resources for blit engine
- With harmonization between all texture types implemented, there is no difference between blit_engine_src and shader_read for supported formats
- Adds extra format filtering to ensure no conflicts when copying data
2019-09-10 16:54:02 +03:00
kd-11 75fcfac00e rsx: Modify find_cached_texture to respect gcm_format. Can pass 0 for "dont care" 2019-09-10 16:54:02 +03:00
Wojtek Arabczyk cb66d05693 Fix warnings in rpcs3/Emu/Cell/Modules/sceNp.cpp 2019-09-09 19:16:02 +03:00
Megamouse aa262889f2 cellVdec: add error_code 2019-09-08 20:54:43 +03:00
Megamouse 6aacec4694 cellMouse: add error_code 2019-09-08 20:54:43 +03:00
Megamouse c111209639 cellGem: add error_code 2019-09-08 20:54:43 +03:00
Nekotekina 0fcddf6568 Audio: restore snd_pcm_recover for ALSA backend
But don't push audio data after recover as before.
2019-09-08 20:49:51 +03:00
Nekotekina aaf1b4c668 Fix gcc warning in bit_cast 2019-09-08 19:27:29 +03:00
Nekotekina 693d54553c Fix compilation 2019-09-08 19:26:10 +03:00
Megamouse cb00b71c4b sceNp: add error_code and some param checks (#6416)
* sceNp: add error_code and basic implementations

* sceNp: and fix some structs

* sceNpSns: improvements

* sceNpUtil: improvements

* sceNpTus: improvements
2019-09-08 18:36:10 +03:00
kd-11 d1603fbb0b vk: Crop malformed image descriptors
- Some image descriptors (lle vdec?) are malformed with pitch being smaller than width
- Crop these for now pending hardware tests
2019-09-08 18:22:27 +03:00
kd-11 f53361b966 rsx: Fix fast texture copy when src_pitch != width * block_size
- Happens on mipmapped linear images
2019-09-08 18:22:27 +03:00
Nekotekina 0c69b7da8f Fix ppu_interpreter::MTFSFI
Out of bounds error.
2019-09-08 14:50:50 +03:00
Nekotekina 798434aada Remove spin wait loop in cpu_thread::check_state
It was mostly dead code.
lock_unlock() method already has spin waiting logic.
2019-09-08 14:50:50 +03:00
kd-11 0af9685381 rsx: Deprecate surface_transform::argb_to_bgra which is no longer required.
- vulkan now uses native swizzle mapping for both surface and texture
2019-09-08 13:56:41 +03:00
kd-11 312bf6840e vk: Fix surface_transform::argb_to_bgra transfers when no scaling is requested 2019-09-08 13:56:41 +03:00
kd-11 cbce309199 vk: Fix depth_stencil scaling 2019-09-08 13:56:41 +03:00
kd-11 48a5cd545f gl: Do not byteswap uint24_8 as it needs a custom 8_24 decoder 2019-09-08 13:56:41 +03:00
Megamouse fa2034087d Input: increase mouse acceleration max to x30 2019-09-08 09:06:16 +02:00
Megamouse cddc045c69 cellPad: disable reserved bit check in cellPadSetActDirect 2019-09-08 08:49:15 +02:00
msuih 5ed700db5d Add cache removal to UI 2019-09-07 22:01:12 +02:00
msuih 84867b0848 Fix deprecation warnings
warning C4996: 'QPalette::Foreground': Use QPalette::WindowText instead
warning C4996: 'QPalette::Background': Use QPalette::Window instead
2019-09-07 22:01:12 +02:00
Eladash 10c81fbdf9 Fix passive reservation locks checking at Emu.Stop() (#6482) 2019-09-07 20:43:16 +03:00
kd-11 440d58f2ff vk: Batch compute jobs when doing texture upload
- Reduces overall number of invocations
2019-09-07 16:23:20 +03:00
kd-11 6aa0b49dbc vk: Prefer using native alignment when uploading.
- Allows using fast copy paths and reduces memory and compute footprint
2019-09-07 16:23:20 +03:00
kd-11 a3a0cb8c17 rsx: Minor texture optimizations 2019-09-07 16:23:20 +03:00
kd-11 efa501dac6 rsx/vp: Set default inputs to (0, 0, 0, 1)
- From some hw tests, it seems this is the default.
2019-09-06 17:08:28 +03:00
kd-11 f8dbe281a5 glsl: Explicitly declare const inputs as such
- Avoids copying the values to temp variables before invoking function calls
- Generates shorter, cleaner AST and SPV bytecode
2019-09-06 17:08:28 +03:00
Kaiwen Xu 68071ddf83 Convert EROFS error 2019-09-05 23:05:30 +03:00
kd-11 14aa3b3360 vk: Remember to allocate enough vertex layout storage objects!
- vertex_layout_storage descriptors were added but the descriptor count was not updated
2019-09-05 19:43:39 +03:00
kd-11 360c0e9af6 vk: Restructure commandbuffer scoping to allow faults in vertex upload
- Defer renderpass open to allow recovery after fault in the middle of vertex upload
2019-09-05 19:43:39 +03:00
Nekotekina d0db7ca6bd cellSaveData fixup (#6393)
Increase entry max size
2019-09-05 14:49:03 +03:00
JohnHolmesII 80d18190f9 Add user message for uppercase RAP 2019-09-05 08:39:11 +02:00
RipleyTom 8de1239132 Simulate BSD FS for file order presentation in savedata 2019-09-05 03:01:46 +03:00
isJuhn aaae437ca3 PPUInterpreter.cpp: remove SSE4.1 requirement in FCTIWZ, FCTID and FCTIDZ 2019-09-04 14:05:33 +03:00
RipleyTom 6bdab5322c Revert File.h encoding. 2019-09-03 08:46:50 +02:00
isJuhn 08c12f4c00 File.cpp: add rounding_alignment argument to get_dir_size to allow GetSizeKB functions to report size more accurately 2019-09-03 03:19:49 +03:00
kd-11 9dc06cef7f rsx: Do not include ro data when attempting to do section merge
- Avoids crazy situations like trying to merge from a 3d or cubemap in memory
2019-09-02 16:49:04 +03:00
Nekotekina a5be410e3f [noexcept] Don't throw on unimplemented HLE functions.
Getting rid of exceptions.
2019-09-02 14:43:31 +03:00
JoãoPaulo 0dfe85dc14 Update FW 4.85 Latest Version (#6437)
* Update FW Latest Version

Updates the latest FW version according to: https://www.playstation.com/en-us/support/system-updates/ps3/

Co-Authored-By: MsDarkLow <msdarklow@gmail.com>
2019-09-01 17:19:11 +01:00
kd-11 e99e8460fe rsx/texture_cache_utils: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 27fabd7607 rsx/ring_buffer: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 33609717f8 rsx/cache: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 f8617500b5 rsx/methods: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 7f7b499303 rsx/util: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 0158a88c88 rsx/textures: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 401bd9112a rsx/prog: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 652f18ebaa rsx/buffers: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 6504daa713 overlays: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 94656ac1e3 rsx/vp: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 0ee9d7b46d rsx/fp: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 756fdedbf6 vk: Warnings cleanup 2019-09-01 18:59:50 +03:00
kd-11 a7b9ff33d8 gl: Warnings cleanup 2019-09-01 18:59:50 +03:00
Megamouse 432364cb04 Qt: add --no-gui mode 2019-08-31 17:08:03 +02:00
Megamouse 7cf037bd49 Fix cli args and app version/name 2019-08-31 17:08:03 +02:00
JohnHolmesII 4f389757ee Update BUILDING.md after #6394 2019-08-31 17:56:45 +03:00
Nekotekina 81a110f346 Fix system time wraparound
Implement utils::udiv128, utils::div128
2019-08-30 22:02:25 +03:00
kd-11 7f99de36c1 rsx: Fixup for surface_target_a flag being broken
- While the mask for surface_a is at index 0, the surface cache expects the order to be maintained correctly!
  Set the correct mask since surface store now checks each RTT individually
2019-08-30 21:46:19 +03:00
kd-11 61af2b7dfc vk: Workgroup tuning for different vendors 2019-08-30 21:46:19 +03:00
kd-11 99fb6d6a5d rsx: Allow GPU-accelerated stream manipulation when doing texture uploads 2019-08-30 21:46:19 +03:00
kd-11 e0a7912d7c rsx: Check for stencil writes when determining zeta_write flag 2019-08-30 21:45:41 +03:00
Ani c4d99f93db rpcs3_version: Bump to 0.0.7 2019-08-30 03:09:59 +01:00
Nekotekina 801adb0947 VFS: Escape Control Characters 0-31
Escaping NUL is formal, it's not expected on input.
2019-08-29 02:44:12 +03:00
Nekotekina b3f5d6d85b Put lv2_obj::awake calls under mutex 2019-08-29 02:44:12 +03:00
kd-11 04c808b8ab rsx: Fixup for MRT color write lookup and surface_target_a 2019-08-28 16:12:10 +03:00
kd-11 e55d216619 rsx: Workarounds for some buggy games
- Replace assert with log message until hardware testing confirms findings
2019-08-28 14:54:51 +03:00
kd-11 e334a43169 rsx: Fix surface cache hit tests
- Avoid silly broken tests due to queue_tag being called before pitch is initialized.
- Return actual memory range covered and exclude trailing padding.
- Coordinates in src are to be calculated with src_pitch, not required_pitch.
2019-08-28 14:54:51 +03:00
JohnHolmesII cca6a19cdd Fix improper string concatenation in rsx_decode. 2019-08-28 01:26:14 +03:00
kd-11 64aff7f019 emucore: Initialize fxo on loading RSX captures
- Fixes crash on capture replay
2019-08-27 21:59:02 +03:00
kd-11 2962e05f26 rsx: Implement per-RTT color masks
- Also refactors and simplifies some common code in surface store and rsx core
2019-08-27 21:59:02 +03:00
kd-11 27aeaf66bc gl: Restructure buffer objects to give more control over usage
- This allows creating buffers with no MAP bits set which should ensure they are created for VRAM usage only
- TODO: Implement compute kernels to avoid software fallback mode for pack/unpack operations
2019-08-27 21:59:02 +03:00
Nekotekina 9672ad53cd Fix SPRX building on AppImage builds 2019-08-27 17:25:55 +03:00
Nekotekina b6b7d7b48c sys_cond_wait fixup 2019-08-27 15:38:18 +03:00
Nekotekina ec8c5453bc Fix sys_cond_wait
Schedule PPU thread before relocking.
Don't lock mutex in notify functions.
2019-08-27 15:13:01 +03:00
Nekotekina fd839243ec Revert PR 6405 2019-08-27 14:05:38 +03:00
Nekotekina ede816387f Fix SPRX loader crash 2019-08-27 13:57:27 +03:00
Nekotekina 796a6646f7 Silence statichle 2019-08-27 13:56:50 +03:00
Nekotekina dd79a5efb7 Remove fxm::make_always 2019-08-27 03:50:15 +03:00
Nekotekina e41b234dc4 Use g_fxo for cellSysCache 2019-08-27 03:50:15 +03:00
Nekotekina d2eba2387b Use g_fxo for display_manager 2019-08-27 03:50:15 +03:00
Nekotekina 6811ec562c Use parentheses for new T() syntax
MSVC keeps being dumb...
2019-08-27 03:50:15 +03:00
Nekotekina fcbf91b7cb Deprecate fxm::get_always
And also its unused fxm::import_always counterpart
2019-08-27 03:50:15 +03:00
Nekotekina 4ec3e968ff Use g_fxo for jit_core_allocator 2019-08-27 03:50:15 +03:00
Nekotekina 3dd6961ee5 Use g_fxo in PPUThread.cpp misc
Replace fxm::get_always
2019-08-27 03:50:15 +03:00
Nekotekina 232e815f17 Init g_fxo in SPRX Loader mode 2019-08-27 03:50:15 +03:00
Nekotekina ebdb0781fa Use g_fxo for statichle_handler 2019-08-27 03:50:15 +03:00
Nekotekina 84e44d3372 Use g_fxo for ppu_linkage_info 2019-08-27 03:50:15 +03:00
Nekotekina 2a37a4bcea Use g_fxo for cellPad
Rename pad_t -> pad_info
2019-08-27 03:50:15 +03:00
Nekotekina 928719b658 Use g_fxo for rsx::avconf 2019-08-27 03:50:15 +03:00
Nekotekina 7a00b5f809 Use g_fxo for patch_engine 2019-08-27 03:50:15 +03:00
Nekotekina cfdd298693 Use g_fxo for cellMusicDecode 2019-08-27 03:50:15 +03:00
Nekotekina 7b0d834477 Use g_fxo for gem_camera_shared 2019-08-27 03:50:15 +03:00
Nekotekina e4a7e8c058 Use g_fxo for cellGem
Rename gem_t -> gem_config
2019-08-27 03:50:15 +03:00
Nekotekina 1d3c18f4e2 Use g_fxo for cellGcm
Rename CellGcmSysConfig -> gcm_config
2019-08-27 03:50:15 +03:00
Nekotekina 9a2338a5ca Use g_fxo for fs_aio_manager (dummy) 2019-08-27 03:50:15 +03:00
Nekotekina 38a06c4b14 Use g_fxo for SysRsxConfig
Rename to lv2_rsx_config
2019-08-27 03:50:15 +03:00
Nekotekina cf44038435 Use g_fxo for lv2_config 2019-08-27 03:50:15 +03:00
Nekotekina 8304043d00 Use g_fxo for sysutil_cb_manager 2019-08-27 03:50:15 +03:00
Eladash a921fe3351 lv2: Wait for rescheduling before confirming ETIMEOUT 2019-08-27 03:44:34 +03:00
Eladash 35a3bed91a sys_cond_wait: Wait for rescheduling before relocking 2019-08-27 03:44:34 +03:00
kd-11 3317e13b64 rsx: Hotfix for semaphore timeout bug
- Add pending flip requests as a reason to invoke the RSX local task handler and release the vblank semaphore
2019-08-26 22:33:29 +03:00
Megamouse 32b5b11a83 cellSaveData/overlays: prevent possible array out of bounds in list view 2019-08-26 21:09:20 +02:00
kd-11 eed32cf3a4 rsx: Decompiler fixups and improvements
- Fix 2D coordinate sampling of W coordinate.
  W is actually HPOS.w and not 1. Z is however always 0.
- Optimize register usage a bit
  Disassembling compiled SPV shows that global declaration results in less ops than using inout modifiers. Modifiers generate extra mov instructions.
2019-08-26 20:03:31 +03:00
kd-11 3e28e4b1e0 rsx/decompiler: Restructure program register behavior
- Fix reading of varying registers in FP
  Different registers have different behavior
- Always write to varying registers. If a register is not written to, it is initialized to (0, 0, 0, 1)
- Reimplements two-sided lighting correctly without hacks
- Also bumps shader cache version
2019-08-26 20:03:31 +03:00
kd-11 fe6ff8622a rsx: Decompiler fixups for conditional execution
- Cond actually obeys vector mask
2019-08-26 20:03:31 +03:00
kd-11 f9aea076ae rsx: Implement depth_buffer_float support.
- Since this is transparent to the application at all time, it only becomes a problem when doing memory transfer or DEPTH->RGBA conversion in shaders.
2019-08-26 20:03:31 +03:00
Ani 0aea33687e appveyor: Update OpenSSL DLLs to 1.1.1
- QT 5.13 requires 1.1.1, default "OpenSSL-Win64" folder on AppVeyor currently points to 1.0.2s
2019-08-26 05:28:56 +01:00
kd-11 9d981de96d rsx: Fix offloader deadlock
- Do not allow offloader to handle its own faults. Serialize them on RSX instead.
  This approach introduces a GPU race condition that should be avoided with improved synchronization.
- TODO: Use proper GPU-side synchronization to avoid this situation
2019-08-25 22:09:20 +03:00
Nekotekina b70908c8f3 Appveyor: use VS 2019 2019-08-22 19:09:06 +03:00
Nekotekina 6165bd68c0 Use g_fxo for avconf_manager 2019-08-22 15:14:33 +03:00
Megamouse 94dd6c2007 cellSaveData: improve param.sfo creation 2019-08-22 08:05:12 +02:00
Megamouse 896cfd2ade cellSaveData/cellMsgDialog: implement cellSaveDataDelete 2019-08-22 08:05:12 +02:00
Megamouse 0b720aeb59 cellSaveData: ignore dot and dotdot 2019-08-22 08:05:12 +02:00
Megamouse 2d29a33ea8 cellSaveData/overlays: initialize with focused entry 2019-08-22 08:05:12 +02:00
Megamouse b3c1759853 cellSaveData/overlays/Qt: fix some warnings and a possible nullptr deref 2019-08-22 08:05:12 +02:00
Eladash ec9b896fbf Fix vm::reserve_map logic 2019-08-22 03:53:40 +03:00
Nekotekina c43a126f62 Fix VFS initialization 2019-08-22 02:46:21 +03:00
Nekotekina 8cf025bf71 Use g_fxo for cellWebBrowser
Rename browser_t -> browser_info
2019-08-22 02:13:39 +03:00
Nekotekina 2b55bde104 Use g_fxo in cellRudp
Rename rudp_t -> rudp_info
2019-08-22 02:13:39 +03:00
Nekotekina 02fe1ef7ff Use g_fxo for cellSearch
Rename search_t -> search_info
2019-08-22 02:13:39 +03:00
Nekotekina 4ac1e2d8f5 Use g_fxo for vfs_manager 2019-08-22 02:13:39 +03:00
Nekotekina 93d5c0a38e Use g_fxo for cellRec
Rename rec_t -> rec_info
2019-08-22 02:13:39 +03:00
Nekotekina 2740fafa64 Use g_fxo for page_fault_notification_entries 2019-08-22 02:13:39 +03:00
Nekotekina 54952f6ad6 Use g_fxo for LoadedNpdrmKeys_t
Rename to loaded_npdrm_keys
2019-08-22 02:13:39 +03:00
Nekotekina a85a8796d3 Use g_fxo for page_fault_event_entries 2019-08-22 02:13:39 +03:00
Nekotekina 606601441d Use g_fxo for cellMusic
Replace music_t with music_state
2019-08-22 02:13:39 +03:00
Nekotekina 2882220cbd Implement cpu_thread::stop_all() 2019-08-22 02:13:39 +03:00
Nekotekina 8517ccfdfa Add new typemap for always existing objects
Not to be confused with singletons or global variables.
2019-08-22 02:13:39 +03:00
Nekotekina 7db2e2537f Don't call lv2_obj::awake_all with empty list
Fixup after #5883
2019-08-22 02:13:39 +03:00
Nekotekina 7a3aa02dc1 StrFmt.h: remove some dead code with UB
Global variable is unused but written concurrently.
2019-08-22 02:13:39 +03:00
kd-11 7c5bde4aeb rsx: Update tag timestamp to match newest inherited data
- Avoids memory appearing older when used for depth test without depth write
  The write_barrier before the call will inherit new data but the tag will not update as no new information is added.
2019-08-21 21:17:15 +03:00
kd-11 c67c97844e rsx: Fixup for blit engine range calculations 2019-08-21 21:17:15 +03:00
kd-11 5d1b7eb945 rsx: Fix reference leaks in texture_cache<->surface_cache communication
- Properly commit orphaned blocks not invalidating existing cache structures
- Do not ignore overwritten objects when commiting as unprotected fbo. Avoids stale references to invalidated surface objects.
2019-08-21 21:17:15 +03:00
kd-11 ca8b0da141 gl: Invalidate range before reading to prevent deadlock 2019-08-21 21:17:15 +03:00
kd-11 141072023b rsx: Fix handling of ARGB8 memory
- Load into memory as straightforward BGRA
- Fixes a bug in vulkan caused by byte shuffling in blit engine vs shader access
- Removes the need for memory shuffling when transferring into a rendertarget
2019-08-21 21:17:15 +03:00
kd-11 9cd5325962 rsx: Free memory 'held hostage' by storage sections in the surface cache
- Once the memory has been captured by another surface, release the allocation
2019-08-21 21:17:15 +03:00
kd-11 be98554b40 rsx: Fix surface split logic
- Calculations are supposed to be done based on the properties of the outgoing surface
2019-08-21 21:17:15 +03:00
kd-11 67dac94704 rsx/fp: Zero-initialize FragDepth register to match hw 2019-08-21 21:17:15 +03:00
kd-11 35e61c77e0 gl: Fixup for D24S8 readback 2019-08-21 21:17:15 +03:00
Megamouse 9c64318e8a Qt: Fall back to VERSION in game list 2019-08-21 07:57:44 +02:00
Alex James b0d0f51d8d crypto: implement AES-NI acceleration
This is based off the upstream implementation in mbedTLS as well as an
external pull request [1] for MSVC support (using intrinsics).

1: https://github.com/ARMmbed/mbedtls/pull/1355
2019-08-19 23:39:40 +03:00
JohnHolmesII bd8cbcdb21 Reapply old template depth to compilers (fixes Gentoo building) 2019-08-19 20:12:31 +03:00
kd-11 dca29def5e rsx: Temporary workaround for race condition in blit engine 2019-08-18 20:45:48 +03:00
kd-11 5e299111cc rsx/vk: Restructure surface access barriers and implement RCB/RDB
- Implements render target data load (aka Read Color Buffer/Read Depth Buffer)
- Refactors vulkan surface barrier to be much cleaner.
- Removes redundant surface barrier invocations after doing a merged load
  from surface cache.
- Adds explicit access modes when gathering surfaces from cache.
2019-08-18 20:45:48 +03:00
kd-11 dfe709d464 rsx: Surface cache restructuring
- Further improve aliased data preservation by unconditionally scanning.
  Its is possible for cache aliasing to occur when doing memory split.
- Also sets up for RCB/RDB implementation
2019-08-18 20:45:48 +03:00
JohnHolmesII 5817e2a359 Fix mismatched enum string formats in sys_config 2019-08-18 18:39:17 +02:00
Silent 87ffa04a2b Improvements to XInput vibration:
- use std::chrono to measure 20ms between XInputSetState calls, because clock() measures cpu time and not real time (so calls were made more often than intended)
- fixup data types for cached vibration values to prevent u8 values overflowing and giving false positives on m_dev->newVibrateData
2019-08-18 18:03:20 +02:00
Megamouse b3aff3a1c6 cellKb: fix large ASCII characters for some compilers 2019-08-18 08:18:15 +02:00
Megamouse eedf96e1f4 Qt: fix diacritics in keyboard handler 2019-08-18 08:18:15 +02:00
Megamouse 7408f50d71 cellKb: improve key conversion 2019-08-18 08:18:15 +02:00
Megamouse 86a8b5924a Add option for keyboard layout 2019-08-18 08:18:15 +02:00
Megamouse 8a31c7a7c8 Qt/input: windows workaround to fix shift on the basic keyboard handler 2019-08-18 08:18:15 +02:00
Eladash 62414cee15 Do not allow SYS_SYNC_PRIORITY_INHERIT in sys_semaphore_create (EINVAL) 2019-08-18 04:01:16 +03:00
Eladash 960c8e6538 Partial compilation fix for WTH_GDB build 2019-08-18 04:01:16 +03:00
eladash 0af342a978 PPU Precise: FPCC handling on float ops implemented 2019-08-18 04:01:16 +03:00
Nekotekina dd86aa472d Fix _sys_lwcond_queue_wait
Fixup after #5883
2019-08-18 00:19:31 +03:00
Nekotekina 6535fb0603 Fix _sys_lwcond_signal_all
Fixup after #5883
2019-08-18 00:19:31 +03:00
Nekotekina 11a43e25d7 typeindices.hpp - start index from 0
Starting it from 1 makes it more error-prone to use.
2019-08-17 23:40:51 +03:00
max 92e6dee60f [rpcs3] refactor includes and additional dependencies 2019-08-17 23:30:23 +03:00
max c5440c95c8 [MSbuild] Extend support to v142 toolset
- Added support for current default toolset, explitly specified c++17 standard and 8.1 win sdk.
  - properties were defined via `common_default.props` and `common_default_macros.props`
 Note: There are 2 files because some properties need to be imported before `Microsoft.Cpp.Default.props` , but other common properties can be imported after but they require marco defined in `Microsoft.Cpp.Default.props`.
- Added hidapi and libusb project files to repository to simplify msbuild changes.
2019-08-17 23:30:23 +03:00
max 4cfff8ea4a [MSbuild] Detect and use MSVC version for NMake CL.
- llvm and glslang projects modified to detect and use Visual Studio Version.
 - Added `-Thost=x64` to llvm NMake CL (warning in cmake lists)
Note: `-A` option is used to set platform since rpcs3 requires cmake 3.8.3+ anyway.
2019-08-17 23:30:23 +03:00
yahfz 1444492eab lv2: Allow sys_sync_priority_inherit 2019-08-17 23:11:08 +03:00
eladash 050339bb3e PPU/LV2: Make thread-lists scheduling atomic 2019-08-17 22:46:36 +03:00
Eladash 500a4fa2fb rsx: Fix potential out of range methods execution (can result in segfaults) 2019-08-17 17:26:04 +01:00
Ani 51188db717 gui: Remove fallback font from YoRHa theme
Starting with QT 5.13, having both fonts causes stylesheets to use the fallback font even if the first font is present, but uses the first font when there's no fallback font.
The fallback font (Arial) can safely be removed as the default one is 'good enough' for the intended purpose of having a fallback.
2019-08-17 01:41:05 +01:00
Eladash 0d88f037ff Add new accuracy control for PUTLLUC accuracy setting (non-TSX)
With the option enabled GET commands are blocked until the current PUTLLC/PUTLLUC executer on that address finishes

Additional improvements:
- Minor race fix of sys_ppu_thread_exit (wait until the writer finishes)
- Max number of ppu threads bumped to 8
2019-08-17 00:42:46 +03:00
Eladash d8402008dd Stub sys_process_get_number_of_object(object == SYS_SPUPORT_OBJECT) 2019-08-16 23:49:24 +03:00
Eladash 0dce6be8cb Implement lv2_spu_image
- Improved sys_spu_image_.. error checking
- sys_process SYS_SPUIMAGE_OBJECT handling implemented
2019-08-16 23:49:24 +03:00
Eladash 80827aa31a Stub sys_process_get_number_of_object(object == SYS_TRACE_OBJECT) 2019-08-16 23:49:24 +03:00
Eladash 6540fecdc3 Add another pattern to ppu analyzer imports 2019-08-16 22:55:04 +03:00
Pierre-Loup A. Griffais f342f4b826 Work around a crash on startup on current versions of KDE on Linux.
https://bugs.kde.org/show_bug.cgi?id=401637
2019-08-16 20:37:41 +03:00
Pierre-Loup A. Griffais 56011cbddd vk: don't die on VK_SUBOPTIMAL_KHR in AcquireNextImage, and recreate swapchain
vkAcquireNextImageKHR can also return VK_SUBOPTIMAL_KHR and is non-fatal.

However, it's a good idea to still recreate the swap chain later to maintain
optimal presentation paths after temporary occlusion.
2019-08-16 20:09:37 +03:00
msuih 42ff93d30c Add advanced tab 2019-08-16 19:58:19 +03:00
Eladash f2a2dfda3a Improve unknown settings values dialog text message 2019-08-16 09:45:24 +02:00
Eladash f0c267ecb0 Change default Sleep Timers Accuracy setting's value to "Usleep Only" 2019-08-16 09:45:24 +02:00
kd-11 a0f0c418d7 gl:Implement proper support for packed 16-bit rendertargets
- Also some minor refactoring
2019-08-15 14:00:17 +03:00
kd-11 7f85b18b46 gl: Add support for 4444 typeless texture 2019-08-15 14:00:17 +03:00
Megamouse 9f854dba98 Qt: move default stylesheet to stylesheets.h 2019-08-14 23:54:09 +02:00
Megamouse 503d43889d rename rpcs3_app to headless_application 2019-08-14 23:54:09 +02:00
Megamouse 0dc7841d0f Add cli arg for hidpi 2019-08-14 23:54:09 +02:00
Megamouse 13367071b0 update some comments 2019-08-14 23:54:09 +02:00
RipleyTom 2040a11585 header shuffling 2019-08-14 23:54:09 +02:00
Megamouse 8debdfcd09 handle empty callback returns 2019-08-14 23:54:09 +02:00
Megamouse 08c581947d Create headless application [WIP] 2019-08-14 23:54:09 +02:00
Nekotekina 4df1c16099 Implement stx::typeinfo_v (util/typeindices.hpp)
Move and generalize logic from typemap.h
2019-08-14 21:04:25 +03:00
Nekotekina 9f9309055f typemap: remove "shared" type support
Need to fix design mistakes of putting too much into the same bag.
2019-08-14 20:28:34 +03:00
Nekotekina d41d7b5e24 typemap: remove "polymorphic" type support
Unnecessary complexity
2019-08-14 20:28:34 +03:00
Nekotekina f8f3067deb Always check page_allocated in vm::check_addr 2019-08-14 20:28:34 +03:00
Nekotekina d45c743877 Remove lib_loading_type::automatic
This is former default and is now obsolete.
2019-08-14 20:28:34 +03:00
Nekotekina 4f22559ff0 Remove dumb overloads of atomic_op, fetch_op
And lf_queue::apply.
Fix inlining problem by passing F by value.
2019-08-14 20:28:34 +03:00
Nekotekina cf16ee5eb5 Implement lib_loading_type::liblv2list 2019-08-14 20:28:34 +03:00
Nekotekina ae46333e99 Implement lib_loading_type::liblv2both 2019-08-14 20:28:34 +03:00
RipleyTom 87bf0386c4 Screenshot function 2019-08-14 19:24:42 +02:00
MarioSonic2987 b3f4cd3a95 Update "Enable native user interface" description
Since the merge of native OSK, back in February 2019, RPCS3 is able to support more languages with native UI. However, the description hasn't been updated since then. 

As far as I know, the languages that are now supported are English, French, German, Italian and Spanish. 
Both Greek and Russian use different alphabets from Latin, therefore they aren't supported. Polish or Norwegian uses special Latin characters that aren't covered in extended ASCII character set. Chinese, Japanese and Korean aren't also supported.
2019-08-14 04:44:59 +01:00
Eladash 7fda07eb5b rsx: UB fix (signed vs unsigned mismatch) 2019-08-13 20:48:50 +01:00
Eladash 519fe9309e rsx: Fix nv0039::buffer_notify 2019-08-13 20:48:50 +01:00
Eladash 527b1bb071 rsx: Fix overlapping transfer of nv3089::image_in when out_pitch != in_pitch
or out_pitch != out_bpp * out_w
2019-08-13 20:48:50 +01:00
Eladash 19825366f5 Fixup for fixup 2019-08-13 17:55:45 +01:00
Eladash a902874b01 Fixup after #6286 2019-08-13 13:34:14 +03:00
Eladash 95f22b76c6 Fixup after #6329 2019-08-13 04:56:00 +03:00
Eladash a4896eb6df Fix race on cellVdecClose TLS cleanup 2019-08-13 04:56:00 +03:00
Eladash 30c72b105e Fix race on notifying vdec thread in cellVdecGetPicture
Can dereference nullptr if cellVdecClose was called simultaneously.
2019-08-13 04:56:00 +03:00
Eladash 14ac70dfbf Avoid using _sys_ppu_thread_exit in _sys_interrupt_thread_disestablish 2019-08-13 04:56:00 +03:00
Eladash 4b82006984 ppu: Improve LWSYNC
Block load<->load reordering as real lwsync.
2019-08-13 04:56:00 +03:00
eladash 4f00af6000 ppu interpreter: Allow non-ssse3 to use fast path
byte shuffle is now removed which was the barrier for ssse3 check.
2019-08-13 04:56:00 +03:00
Eladash 6d3fc3a386 core config: Expose min/max ranges of integral settings and use it 2019-08-13 04:56:00 +03:00
Eladash a6c94a0eaf Fix possible infinite loop on vm area searching (sys_mmapper_allocate_address)
Specifically when allocation with 0x8000'0000 alignment fails.
2019-08-13 04:56:00 +03:00
Eladash cd4e895c04 Use error_code in mmapper_thread_recover_page_fault 2019-08-13 04:56:00 +03:00
Eladash 629d8e3a51 Fix sys_time_get_current_time() date 2019-08-13 04:56:00 +03:00
Eladash 6d0f2ae58a ppu interpreter: Improve SRD/SLD 2019-08-13 04:56:00 +03:00
Eladash a560498cd4 ppu: Improve FCTIW, FCTIWZ, FCTID and FCTIDZ 2019-08-13 04:56:00 +03:00
Eladash d48d424b19 Return immediatly from fs::remove_all on failure to remove entries
This way error code will be reflected properly
2019-08-12 14:37:23 +03:00
Eladash 0a5b6ad928 Add missing EALIGN check for sys_mmapper_search_and_map 2019-08-11 21:43:13 +03:00
Eladash a832581201 Fix sys_vm address boundary checks (fix vsize) 2019-08-11 21:43:13 +03:00
Eladash ecb3b62787 Fix CELL_CAMERA_ERROR_NOT_INIT check in cellCameraSetNotifyEventQueue when camera set to null
Don't worry, this was a duplicated check for null camera setting.
2019-08-11 21:43:13 +03:00
Eladash 96527eee5b idm: Fix bug in ID searching
* ID checking now checks if (id % step) == 0
* Extend possible ID range to allow UINT32_MAX (as long as base is non-zero)
2019-08-11 21:43:13 +03:00
Eladash 5c904bf3e2 Fix max allocations for DECR mode (sys_memory) 2019-08-11 21:43:13 +03:00
Eladash cbcd06d1dc ppu: Stack size allocation improvements 2019-08-11 21:43:13 +03:00
Eladash 3ce18fd960 Implement vm::page_executable (#6330)
Fixes segfaults when attenpting to set segfaults on non-executable memory.
2019-08-11 21:04:17 +03:00
Eladash 94d33003a3 Write zero in attr->pad (sys_memory_get_page_attribute)
Its included in lv2 as well.
2019-08-10 19:47:37 +03:00
Eladash 25e47e0de0 Fix vm::check_addr memory state check 2019-08-10 19:47:37 +03:00
kd-11 8866a3d6a9 rsx: Cleanup for blit engine fixes 2019-08-10 16:45:02 +01:00
msuih 8150095e8f Qt warnings cleanup 2019-08-10 16:15:24 +02:00
Megamouse f051f53e15 Qt: add setting for trophy manager icon background color 2019-08-10 15:13:36 +02:00
Megamouse 2f12226ff9 Qt: add setting for save manager background icon colors + fix icon size 2019-08-10 15:13:36 +02:00
Megamouse d4c635a83c Qt: change default icon background color to something friendlier looking 2019-08-10 15:13:36 +02:00
Megamouse 697128464b Qt: add some sanity checks to prevent list crashes 2019-08-10 15:13:36 +02:00
Megamouse 0559e929d4 fix some warnings 2019-08-10 15:13:36 +02:00
Megamouse efcb46658b Qt: fix initial trophy icon background color 2019-08-10 15:13:36 +02:00
Megamouse 01720afba9 silence some warnings 2019-08-10 11:02:20 +02:00
Megamouse e21b8eb95b Update Appveyor to Qt 5.13 2019-08-10 11:02:20 +02:00
kd-11 033836d88c rsx: Minor fixup for nv3089::image_in
- Typo scale_x->scale_y
- Remove convoluted temp buffer creation and just use vector instead
2019-08-08 15:48:22 +03:00
Eladash 112ad9dda2 Allow to load liblv2.sprx with manual/both lib loading options (#6274) 2019-08-07 18:41:27 +03:00
RipleyTom 05813c7e90 Fixes suspend+stop threads not stopping 2019-08-04 21:16:19 +03:00
xddxd 6e89e4ebbf Update game_list_frame.cpp 2019-08-04 13:29:45 +02:00
Megamouse 4ad784d614 Qt: select custom configs when opening the custom config folder 2019-08-04 08:45:52 +02:00
Eladash c219417f62 sys_vm: Fix sys_vm_invalidate 2019-08-03 14:29:17 +01:00
Nekotekina 5bd17a44c9 Add fallback implementation for waitable atomics
May improve perf on OSX/BSD
2019-08-02 18:37:23 +03:00
Eladash 72fb3ba794 perf hotfix for sys_timer_usleep 2019-08-01 23:38:09 +03:00
kd-11 f0bd0b5a7c rsx: Conditional render sync optimization
- ZCULL queue was updated to one-per-cb but the conditional render sync hint was not updated.
- Do not unconditionally flush the queue unless the upcoming ref is contained in the active CB.
- This avoids spamming queue flush, which frees up resources and improves performance
2019-07-30 21:13:42 +03:00
Malcolm Jestadt d689a6e47b vk: Don't warn RADV users on LLVM 8.0.1
- The 'back screen' issue on RADV was resolved with LLVM 8.0.1
2019-07-30 19:56:05 +03:00
eladash e40b76b253 sys_lwcond: Extract protocol from lwmutex at creation
This is the only reason theres a need to specify lwmutex_sq id at creation. unlike sys_cond, lwcond isn't connected to lwmutex at the lv2 level.
SYS_SYNC_RETRY fix is done explicitly at the firmware level.
This fixes issues when the original lwcond and lwmutexol data got corrupted or deallocated, this can happen when the program simply memcpy the control data to somewhere else.
Or if it uses direct lv2 the lwcond conrol param can even be NULL which will make it access violate when dereferncing it. (This param is unchecked and can be anything)
2019-07-29 21:58:04 +03:00
eladash 7508ae6e65 Add EAGAIN check for sys_memory_contianer_create 2019-07-29 21:55:59 +03:00
eladash 1e5e0f3b5e make sys_fs_closedir atomic 2019-07-29 21:55:59 +03:00
Nekotekina 949f33c8a4 Fix sys_config_get_io_event syscall name 2019-07-29 21:55:59 +03:00
Nekotekina c6993d3f1c Change bits of waitable atomics
Reduce max waiter count to 2^14.
Refactor code to use waiter_mask and signal_mask constants.
2019-07-29 21:55:59 +03:00
Nekotekina e1db6cef6f Remove ignored bits in waitable atomics
It was not implemented correctly.
2019-07-29 21:55:59 +03:00
Eladash 829047ecdb ppu disasm: Improve ORI and ORIS disassembly 2019-07-29 18:28:39 +03:00
Nekotekina 8886414013 Use atomic wait in shared_mutex and semaphore 2019-07-29 03:04:55 +03:00
Nekotekina f63e89f9b4 Implement waitable atomics
Moved Atomic.h to util/atomic.hpp
List source files in CMakeLists.txt
2019-07-29 03:04:55 +03:00
Nekotekina ec2db8edbc Correct get_int_t to get_uint_t.
Add get_sint_t.
2019-07-29 00:12:07 +03:00
Eladash 997e3046e3 vm/sys_overlay Improvements
- Implement sys_overlay_load_module_by_fd.
- Implement special segment allocation when ppc_seg flag is specified.
2019-07-28 14:23:58 +03:00
Megamouse 65134f73d6 Qt: properly scale icons in trophy manager 2019-07-27 15:29:05 +02:00
Megamouse 76a1fd5c88 Qt: simplify game list icon resize 2019-07-27 15:29:05 +02:00
kd-11 1de90bdb1f rsx: Improve aliased data preservation
- Carve out inherited region if any
- Perform pitch compatibility test before assigning old_surface
2019-07-27 16:09:21 +03:00
msuih 738cfccbba Save manager improvements 2019-07-27 11:48:37 +02:00
Eladash 230c3d55b6 Fixup 2019-07-27 04:03:29 +01:00
Eladash fcc75c8b0f rsx: Write atomically semaphore updates and fix zcull timestamp 2019-07-26 21:27:55 +03:00
Eladash 22994d15fb Fix cellCamera events support with clocks scaling 2019-07-26 21:27:55 +03:00
Eladash c53f0dd7b5 rsx: Fix gcm unmap events 2019-07-26 21:27:55 +03:00
Megamouse b56b425b19 Emu: set m_title to the actual disc game title 2019-07-25 08:53:07 +02:00
Megamouse b44b44b3c4 Qt: cleanup some comments 2019-07-25 08:53:07 +02:00
Megamouse 71c56b719c Emu/overlays: fix background picture path 2019-07-25 08:53:07 +02:00
Nekotekina 090c71aa7c TSX: Improve cpu_thread::suspend_all implementation
Implement low_lock and vip_lock (for shared_mutex).
Try to simplify suspend_all implementation with updated shared_mutex.
2019-07-24 17:25:52 +03:00
Eladash 49aefc0795 Prefetch MFC list elements (#5345)
* Prefetch mfc list elemets to protect from overwriting

Also move some stuff away from command processing such as a few constant arguments setup
2019-07-24 01:13:45 +03:00
Eladash 85b1152e29 Timers scaling and fixes 2019-07-23 00:09:01 +01:00
Eladash d17be2c2ec Fix similar bug in sys_rsx_context_iomap
TODO: Is there an rsxmem unmap event for those cases?
2019-07-22 15:16:43 +03:00
Eladash cf4d715284 typo fix (to squash) 2019-07-22 15:16:43 +03:00
Eladash 2b721b9e29 correctness fix
Not that it really matters because of gcm's lwmutex but just in case the syscall is useb directly.
2019-07-22 15:16:43 +03:00
Eladash 492c9e3c7c Fix sys_rsx_context_iounmap partial unmapping 2019-07-22 15:16:43 +03:00
Rui Pinheiro 070c3af50f Initial sys_config implementation 2019-07-22 02:44:45 +03:00
Zangetsu38 a98a2b79d0 thread scheduler: Add scheduler for zen2 and refactor for zen(+). 2019-07-21 23:02:11 +03:00
Megamouse 6128fffd3e Qt: make text labels in about dialog selectable 2019-07-21 20:07:19 +02:00
Silent 66c1143a65 Improvements to Virtual File System dialog:
- Replace Add Directory and Reset buttons with + and - buttons
- Add a confirmation message before Reset All
- Rename "Okay" to "Save" (to be in line with the rest of the UI) and add a Close option to quit without savin
2019-07-21 13:25:43 +02:00
Silent f5f0a5aa19 Use move semantics with several shared pointers in vfs_dialog, they need not be copied 2019-07-21 13:25:43 +02:00
Megamouse 78ecacabcb input: keep pads intercepted while regular buttons are still pressed 2019-07-21 10:18:59 +02:00
kd-11 9a7c2784f0 rsx: Do not clip scissor to viewport when doing buffer clear 2019-07-20 16:39:32 +03:00
Nekotekina 4e4c896136 Expand thread affinity mask to u64
Also fix and note __APPLE__ path.
2019-07-20 15:58:28 +03:00
kd-11 e2574ff100 rsx: Support CSAA transparency without multiple rasterization samples enabled 2019-07-19 15:49:08 +03:00
Malcolm Jestadt 79ddf55937 Fix decryption filtering for case sensitive platforms
- Filter for .BIN instead of .bin
2019-07-19 14:46:53 +03:00
kd-11 b5a2f0df68 rsx: Implement separate viewport raster clipping
- Merge viewport raster window and scissor into one clipping region
- Viewport raster clip is different from viewport geometry clipping in
hardware as the latter is configurable separately
2019-07-19 14:21:19 +03:00
Nekotekina 1b140c8e97 SPU Recompiler: optimize JIT memory consumption
Avoid rebuilding trampoline for every function at startup.
This should fix Out of Memory error in some cases.
2019-07-19 04:56:35 +03:00
msuih 95b6883ad4 Tiny typo fix 2019-07-18 19:16:17 +02:00
msuih d9c311737d Keep custom name when adding/removing game-specific config 2019-07-18 19:16:17 +02:00
scribam a268415121 vk: Use macros from Vulkan SDK 2019-07-17 17:56:29 +03:00
kd-11 ea2f4d57fa rsx: Fixups 2019-07-17 13:29:42 +03:00
kd-11 113a49e00c rsx: Handle cyclic references when doing memory inheritance 2019-07-17 13:29:42 +03:00
kd-11 34b06453f9 rsx: Handle lost data due to unused data sections
- After splitting, the sections may not be referenced at all for anything other than just pixel storage
- In such cases, either merge down or sample from the upstream source instead
2019-07-17 13:29:42 +03:00
kd-11 998717659f rsx: Fix reference leak when cloning surfaces 2019-07-17 13:29:42 +03:00
kd-11 009e01a347 rsx: Set up for multi-section inheritance 2019-07-17 13:29:42 +03:00
Alex James 397e5dd1ea Qt/game_list_frame: Ignore file entries when adding disc/game directories
Fixes #6212.
2019-07-16 15:58:52 +03:00
Nekotekina 736ec4035e Fixup for SPU Recompilers, regression after #6210 2019-07-15 19:51:58 +03:00
Nekotekina c062000288 Add experimental TSC frequency detection 2019-07-15 17:20:12 +03:00
Nekotekina cb5c26f2b5 Fix SPU Interpreter regression after #6147 2019-07-15 16:34:34 +03:00
Eladash 1e2f81dd33 kernel explorer typo fix 2019-07-15 15:50:40 +03:00
Nekotekina c01f1a8968 Avoid transitive include of vm_ref.h
Add forward declarations of vm::_ref_base
Remove default AT = u32 in _ptr_base and _ref_base (doesn't play well).
2019-07-15 15:46:46 +03:00
Nekotekina cfa1416d64 sys_spu: add vm::temporary_unlock 2019-07-14 18:33:23 +03:00
Nekotekina 1a1eff66bc sys_timer: add vm::temporary_unlock 2019-07-14 18:21:56 +03:00
Nekotekina bc1617758c sys_semaphore: add vm::temporary_unlock 2019-07-14 18:18:03 +03:00
Nekotekina 6126a98111 sys_rwlock: add vm::temporary_unlock 2019-07-14 18:14:15 +03:00
Nekotekina e1293b14f8 sys_net: add vm::temporary_unlock 2019-07-14 18:11:00 +03:00
Nekotekina f5ee8c1292 sys_interrupt: add vm::temporary_unlock 2019-07-14 18:08:18 +03:00
Nekotekina 0a88a23bf5 sys_event: add vm::temporary_unlock 2019-07-14 18:06:02 +03:00
Nekotekina cbfa6cc91f sys_vm: add vm::temporary_unlock 2019-07-14 17:55:26 +03:00
Nekotekina 38d4e3c184 sys_mmapper: add vm::temporary_unlock 2019-07-14 17:51:03 +03:00
Nekotekina 7398cfc950 sys_event_flag: add vm::temporary_unlock 2019-07-14 17:37:58 +03:00
Nekotekina 22e4ef147a SPU TSX: fix "Preferred SPU Threads" 2019-07-14 17:33:20 +03:00
Megamouse 9ff6486c85 Qt: Fix small controller icon in pad settings for high dpi modes 2019-07-14 16:19:02 +02:00
Eladash c727760537 Fix g_exec_addr addres check for access violations 2019-07-13 16:59:58 +03:00
Eladash 3de885c7a6 Dealloc exec cache after sys_prx/overlay_unload_module
Must come before vm::dealloc of it.
2019-07-13 16:59:58 +03:00
Megamouse 3d6c211382 Qt: fix app versions after a patch was removed from the game list 2019-07-13 10:50:41 +02:00
Malcolm Jestadt 94af3b3f03 vk: Fix Linux Vega float16_t workaround
- It was disabling float16_t for non Vega cards
2019-07-12 12:25:46 +03:00
yahfz ae055f6792 Set the default SPU Decoder to LLVM and rename SPU LLVM to "fastest" 2019-07-11 23:39:27 +02:00
Eladash 17c8ac9ab8 rsx: Debugger output text fix 2019-07-12 00:19:56 +03:00
Eladash c4d8ef4340 rsx: Allow to configure vblank rate
Removed "HLE protection" hack from sys_rsx_context_attribute
2019-07-12 00:19:56 +03:00
Eladash efbae02a22 Fix Emu.Stop() with vulkan device enumeration hack 2019-07-11 23:34:29 +03:00
Silent f3551cedb7 rsx: Swap R and B channels in SET_BLEND_COLOR since this color is BGRA, not RGBA 2019-07-11 22:51:01 +03:00
RipleyTom 6c6b973342 Microphone implementation 2019-07-11 20:13:12 +01:00
kd-11 2898309f68 vk: Silence some debug prints
- This message confuses some users
2019-07-11 13:22:13 +03:00
kd-11 fc09572648 rsx: Implement texel border decode
- Texel borders are no longer actually supported in modern APIs
- Removes the border texels and uses border color instead which is incorrect but should work fine
2019-07-11 13:22:13 +03:00
kd-11 d8f753f1e8 rsx: Do not allow framebuffer surfaces that exceed their allocated pitch dimensions
- Truncate surfaces to forcefully fit inside the declared region
2019-07-11 13:22:13 +03:00
Eladash 537d3f2548 Log last function on debug pause or exception, dump cpu_thread state on access violation 2019-07-10 17:35:39 +03:00
Eladash d7a2d42d8f MFC: Fix Tag Status report for sync/eieio/barrier commands 2019-07-10 17:35:39 +03:00
eladash 0979207cc6 Log currently PPU HLE function executed after access violation
old functionality retrieved
2019-07-10 17:35:39 +03:00
Silent 2016a33277 Fixed behaviour of buttons in Welcome, Find, Settings, VFS and Debugger dialogs - now reacting to button click, not press 2019-07-10 13:14:05 +03:00
Margen67 ab228d6407 Fix #6180 (typo in UI options) 2019-07-10 07:55:01 +02:00
Eladash 78e447e28c rsx: Typo fix 2019-07-09 22:47:55 +03:00
kd-11 2548057ea0 vk: Improve AMD driver support
- Workaround broken fp16 in AMDVLK/RADV
- Do not disable primitive restart as the issue seems to have been fixed
2019-07-09 16:27:59 +03:00
kd-11 956270d9be gl: Add readback/writeback config for format GL_R16 2019-07-09 16:27:59 +03:00
kd-11 c072c511a1 rsx: Add support for slice padding rows when gathering slices for cubemap/3d 2019-07-09 16:27:59 +03:00
kd-11 9ca6546dec vk: When reusing resources, make sure to reinitialize the component layout 2019-07-09 16:27:59 +03:00
kd-11 0cc672dcb3 vk: "Improve" initialization hack
- Change default alpha to 1 from 0
- TODO: Implement memory tagging for synchronizing this
2019-07-09 16:27:59 +03:00
kd-11 219a5382f7 rsx: If no array streams are enabled, mark inline array as disabled (null render) 2019-07-09 16:27:59 +03:00
kd-11 7840cd914e rsx: Fixup nv3089::image_in
- Correct pitch when sourcing from temp block
- Remove obsolete? double transfer that also introduced a stale pointer reference to freed memory
2019-07-09 16:27:59 +03:00
kd-11 c47f4fd59e vk: Fix frame skipping 2019-07-09 16:27:59 +03:00
Nekotekina b9130dd663 Remove redundant const on return value in rsx_methods.h 2019-07-09 12:09:21 +03:00
Nekotekina d1db6a6827 Revert incorrect usage of le_t in name_u64 2019-07-09 12:07:58 +03:00
Eladash 6fba3dd90c Misc correctness improvement
Use let_<u64> for name_64 and prefer it when possible.
2019-07-09 11:52:34 +03:00
Eladash 9d9e158b9a Make more use of the new get_int_t 2019-07-09 11:52:34 +03:00
Eladash d57b4dc8f3 rsx: Refactor rsx_decode.h and bugfixes 2019-07-09 11:52:34 +03:00
Nekotekina db4bc6f6be Add RESTRICT macro
Applied as C keyword 'restrict'
2019-07-08 19:43:16 +03:00
Nekotekina 92c08be387 Build with -fno-strict-aliasing
Doesn't affect MSVC build since MSVC already works in a similar manner.
2019-07-08 19:39:44 +03:00
kd-11 50736263d2 gl: Fix native pitch computation 2019-07-08 18:04:56 +03:00
RipleyTom 20dd7c0200 Adds spurs_test.self to test bins 2019-07-08 05:37:46 +01:00
arabek b6cfa5c9b9 Fix build with cotire and clang-8.0 branch
Without this patch, build fails when using
cotire, because pch headers are missing
the pthread flag when using Clang
2019-07-08 03:38:18 +01:00
Eladash 6d65d3424f rsx: Clamp fragment shaders address 2019-07-06 20:58:18 +03:00
Eladash 4c2fb54b99 Fix possible inconsistencies for sys_memory mem stats report 2019-07-04 22:35:22 +03:00
kd-11 ad10eb391e vk: Reuse discarded memory whenever possible instead of recreating new
objects
- Memory allocations are surprisingly expensive when spammed
2019-07-03 15:52:16 +03:00
kd-11 71e809a78b rsx: Implement dma abort in case of a reset after misprediction 2019-07-03 15:52:16 +03:00
kd-11 0f11939faf vk: Refactor gc 2019-07-03 15:52:16 +03:00
kd-11 ae93b417ec vk: Handle emergency commandbuffer close with dangling queries
- TODO: Refactoring
2019-07-03 15:52:16 +03:00
kd-11 d69e8288ad vk: Restructure commandbuffer submission into tagged event IDs
- Tagged eventIDs can be used to safely delete resources that are no
longer used
- TODO: Expand gc to collect images as well
- TODO: Fix the texture cache to avoid over-allocating image resources
2019-07-03 15:52:16 +03:00
kd-11 ce04a797c3 vk: Fix event signal race when speculation fails to avoid a cache miss
- TODO: Proper GC for stale events
2019-07-03 15:52:16 +03:00
Malcolm Jestadt b5d5113803 gl: Workaround slow PBO usage with Mesa
-Mesa is currently fastest with GL_STREAM_COPY
-See https://gitlab.freedesktop.org/mesa/mesa/commit/a338dc01866ce50bf7555ee8dc08491c7f63b585
-Also see https://bugs.freedesktop.org/show_bug.cgi?id=111043
2019-07-03 11:28:29 +03:00
msuih 690cdff0d3 Minor fixes
- Fix a typo in OpenAL
- Fix typo in cellHttp.h
- Unused variables in catch
- Use 64-bit shifts
- Use use_count with shared pointers, unique is depracated and getting removed
- Explicitly cast boolean to int
- Signed/unsigned issues with loop variables
- Fix missing return statement (the code path is unreachable, but compiler wants a return)
- */ ouside of comment
- Fix duplicate layout name
2019-07-01 04:33:23 +03:00
msuih 1e0289bcb2 Update libusb 2019-07-01 04:33:23 +03:00
msuih 503e023815 Explicitly downcast integers 2019-07-01 04:33:23 +03:00
msuih d57124d075 Explicitly cast size_t to integer types 2019-07-01 04:33:23 +03:00
msuih 146e43b6ec Do not use negative unsigned literals 2019-07-01 04:33:23 +03:00
msuih 13a557fe67 Silence some warnings
These are technically UB but work just fine
2019-07-01 04:20:00 +03:00
msuih 3d8389457e Disable more 3rdparty warnings 2019-07-01 04:20:00 +03:00
Eladash 2bce367488 Fixup for fixup (#6153)
* Fixup for fixup

* Fix memory ordering for MTRSX

volatile doesnt block reordering.

* ugh
2019-06-30 12:47:42 +03:00
Eladash 43f919c04b Fixup after #6143 (#6146)
vm::spu max address was overflowing resulting in issues, so cast to u64 where needed. Fixes #6145.
    Use vm::get_addr instead of manually substructing vm::base(0) from pointer in texture cache code.
    Prefer std::atomic_thread_fence over _mm_?fence(), adjust usage to be more correct.
    Used sequantially consistent ordering in semaphore_release for TSX path as well.
    Improved memory ordering for sys_rsx_context_iounmap/map.
    Fixed sync bugs in HLE gcm because of not using atomic instructions.
    Use release memory barrier in lwsync for PPU LLVM, according to this xbox360 programming guide lwsync is a hw release memory barrier.
    Also use release barrier where lwsync was originally used in liblv2 sys_lwmutex and cellSync.
    Use acquire barrier for isync instruction, see https://devblogs.microsoft.com/oldnewthing/20180814-00/?p=99485
2019-06-29 18:48:42 +03:00
Eladash 1ee7b91646 Refactoring (#6143)
Prefer vm::ptr<>::ptr over vm::get_addr.
    Prefer vm::_ptr/base over vm::g_base_addr with offset.
    Added methods atomic_t<>::bts and atomic_t<>::btr .
    Removed obsolute rsx::thread::Read/WriteIO32 methods.
    Removed wrong check in semaphore_release.
    Added handling for PUTRx commands for RawSPU MFC proxy.
    Prefer overloaded methods of v128 instead of _mm_... in VPKSHUS ppu interpreter precise.
    Fixed more potential overflows that may result in wrong behaviour.
    Added io/size alignment check for sys_rsx_context_iounmap.
    Added rsx::constants::local_mem_base which represents RSX local memory base address.
    Removed obsolute rsx::thread::main_mem_addr/ioSize/ioAddress members.
2019-06-29 01:27:49 +03:00
msuih e043412be4 Add timeout for vulkan device enumeration 2019-06-28 23:34:13 +03:00
JohnHolmesII 232a35b6fc Various small warning fixes
-Indentation warnings
-prevent shift overflow
-This was declared extern in all contexts. Remove this for initialization
-Fix main return types. OH CANADA!
-Silence extraneos 'unused expression' warning
-Force use return value (warning)
-Remove tautological compare copy-pasta (char always < 256)
2019-06-28 01:45:29 +03:00
JohnHolmesII 948c1df969 Remove unecessary vulkan loader check var, per kd 2019-06-28 01:45:29 +03:00
JohnHolmesII a124ec4a26 Remove braces around shader source strings (warnings) 2019-06-28 01:45:29 +03:00
JohnHolmesII d0eae7bab1 Fix -Wsign-compare a little bit
Explicitly mark loop types (per review)
2019-06-28 01:45:29 +03:00
JohnHolmesII 22917084d9 Explcitly mark overflow in various SIMD functions. Doing so silently
created warnings.
2019-06-28 01:40:52 +03:00
JohnHolmesII ebb1ae6408 Properly ignore SIMD macros to avoid warning 2019-06-28 01:40:52 +03:00
JohnHolmesII 23094b48bb Fix warnings related to -Wswitch
Add default cases.
Move default breaks to newline
Add proper handling in some instances.
Add missing enums to switches
2019-06-28 01:40:52 +03:00
JohnHolmesII be521ff0ab Fix warnings related to parentheses 2019-06-25 20:36:32 -07:00
JohnHolmesII 32f8f40a4a Disable warnings for stb_image.cpp. Should this file be moved to
3rdparty?
2019-06-25 20:36:32 -07:00
JohnHolmesII 1fcf62fdf0 asmjit cleaned up their cmake warnings (no code change) 2019-06-25 20:36:32 -07:00
JohnHolmesII 27fca2ed33 CMake refactor 2019-06-25 20:36:32 -07:00
Nekotekina 1641be5e0c Fix UTF-8 BOM in vm.cpp 2019-06-25 22:21:56 +03:00
Malcolm Jestadt 31afd046b0 lv2: sys_timer_usleep improvements for linux
-The minimum quantum on linux appears to be 50 microseconds by default, not 100
-Do not wait for the last quantum to avoid sleeping too long
2019-06-25 20:20:05 +01:00
kd-11 9ce7b8a401 vk: Add LLVM8 warning for RADV drivers 2019-06-25 20:50:54 +03:00
kd-11 009c55fcba vk: Fix broken layout stream on first draw call 2019-06-25 20:50:54 +03:00
kd-11 4ff77a8555 rsx: Improve balancing of the offloader thread
- Use two counters to avoid atomic operations
- Yield instead of sleeping because some games are very sensitive to timing
2019-06-25 20:50:54 +03:00
kd-11 8249d51aa8 vk: Optimize occlusion pool management
- Do not consume a slot every draw call, instead batch as many draws as possible
- Since renderpasses are dispatched per-draw-clause, keeping occlusion queries outside the renderpasses works fine
- If renderpasses are reorganized, occlusion tasks will have to be reorganized again
2019-06-25 20:50:54 +03:00
kd-11 1ee675e1f4 facepalm of the year
- Typo fix
- This check leads to forever relocating memory if size never exceeds capacity!
2019-06-25 20:50:54 +03:00
kd-11 2b9c315374 rsx: Use rpcs3 thread construct for the offloader thread 2019-06-25 20:50:54 +03:00
kd-11 d26b25816d rsx: Improve profiling setup
- Avoid spamming QPC when not needed
- Free performance when debug overlay is not enabled
2019-06-25 20:50:54 +03:00
kd-11 b893a75002 rsx: Rework RSX offloading
- Use a lockless queue
- Do not enqueue small transfers
2019-06-25 20:50:54 +03:00
kd-11 c32c1b0a62 gl: Minor API tweaks
- Avoid spamming the driver with samplerParameter calls unless the parameters have actually changed
2019-06-25 20:50:54 +03:00
kd-11 6a32f716db rsx: Reimplement vertex layout streaming
- Remove string comparisons from the hot-path!
- Use attribute streaming and push constants to avoid forcing a descriptor block copy every other draw call/pass.
  While this isn't so bad on nvidia cards, it makes AMD cards a slideshow.
2019-06-25 20:50:54 +03:00
kd-11 59ee74a275 rsx: Disable vertex cache if multithreaded memory access is enabled
- When multithreaded RSX is enabled, the vertex cache just lowers performance
- The small cost of upload is paid by the asynchronous thread, allowing RSX to work optimally
2019-06-25 20:50:54 +03:00
kd-11 0fa3bcc336 rsx: Asynchronous data transfer 2019-06-25 20:50:54 +03:00
kd-11 358169507c rsx: Use SSE to accelerate index buffer uploads 2019-06-25 20:50:54 +03:00
kd-11 b645ebdb04 vk: Refactor device management and improve driver detection 2019-06-25 20:50:54 +03:00
kd-11 25bba9bf56 vk: API update
- use KHRONOS_validation instead of LUNARG_standard_validation which is deprecated
2019-06-25 20:50:54 +03:00
kd-11 f113cfe5c0 vk: Avoid some useless memory barriers
- Do format conversions only when necessary
2019-06-25 20:50:54 +03:00
kd-11 c9501b60ab rsx: Use explicit fma for MAD emulation 2019-06-25 20:50:54 +03:00
kd-11 6be7c58fa4 glsl: Refactoring, cleanup and optimizations
- Avoid generating unused code
- Reduce GPR usage in emitted code
2019-06-25 20:50:54 +03:00
Lassi Hämäläinen c963c51a60 Remove unnecessary header includes
- Manually removed lot of unneeded #includes to clean code and reduce
  compilation time
- Reordered some of the #includes to be in more logical order
2019-06-25 17:11:10 +03:00
Lassi Hämäläinen 23432d420d CMake: Disable RTTI for jit.cpp on MSVC 2019-06-25 17:11:10 +03:00
Lassi Hämäläinen a070a414a6 Move rsx::constants and rsx::limits to rsx_utils.h 2019-06-25 17:11:10 +03:00
Lassi Hämäläinen e9e87b8bd9 Add missing #includes to header files
- Multiple header files where missing #includes to other headers that
  where used in the header. Correct header was included in correct
  order in source files which caused everything to compile.
- Added missing #includes so header files correctly include all their
  dependencies and fixes problems with IDEs being unable to parse
  headers correctly due to missing symbols
2019-06-25 17:11:10 +03:00
Lassi Hämäläinen 499035512b Split Emu/Memory into more logical headers
- Add vm_locking.h and vm_reservation.h and move relevant functions
  and types to these headers.
- Change include order and make vm_ptr.h, vm_var.h and vm_ref.h headers
  usable invidually and them including vm.h instead of other way around
- Because usage of vm::ptr now requires including vm_ptr.h instead of
  vm.h updated multiple #includes
- Added additional #includes to vm_reservation.h and vm_locking to
  where vm::reservation_* and locking related functions are used
2019-06-25 17:11:10 +03:00
Megamouse 8e942eb50c Qt: add more filters to the decryption file dialog and fix file endings 2019-06-23 21:08:58 +03:00
Eladash 806a7bbf04 Fixup for #6115 (#6120) 2019-06-22 12:10:47 +03:00
Eladash cd0ef99df5 Fix BE endianess arch support in semaphore_406e (#6116)
Add raw() methods for endianness support types and make use of it.
2019-06-21 19:29:49 +03:00
Eladash ade291e73d Fix potential overflow in sys_vm 2019-06-21 00:02:52 +03:00
Nekotekina 517a2bc34a sys_lwcond: add vm::temporary_unlock 2019-06-20 14:46:32 +03:00
Nekotekina a935203a18 sys_lwmutex: add vm::temporary_unlock 2019-06-20 14:46:32 +03:00
Nekotekina b9b591bf02 Fix SPU Loop Detection 2019-06-20 14:46:32 +03:00
Megamouse 3602d45cb7 input: raise max mouse acceleration to x10 2019-06-20 10:23:50 +02:00
Megamouse a0f2ade55d gui/input: improve log message for GetNextButtonPress 2019-06-20 10:23:50 +02:00
Nekotekina 5753b3222e sys_memory: add vm::temporary_unlock 2019-06-19 20:45:59 +03:00
Nekotekina d021d9e14a sys_mutex: add vm::temporary_unlock 2019-06-19 20:39:39 +03:00
Nekotekina 89a31292dc sys_cond: add vm::temporary_unlock 2019-06-19 20:39:29 +03:00
Nekotekina 5d45a3e47d Implement cpu_thread::suspend_all
Remove Accurate PUTLLC option.
Implement fallback path for SPU transactions.
2019-06-19 20:36:12 +03:00
Megamouse 17d0dcb7a2 gui/input: add evdev callback when no button was pressed 2019-06-18 18:15:19 +02:00
Eladash d52953fca6 minor UB fix 2019-06-17 02:52:43 +03:00
Ivan 0fe26f8293 Add sys_rsx_context_iomap workaround
Don't fail with CELL_EINVAL on vm::main.
2019-06-17 01:17:02 +03:00
scribam 185fd3d257 rsx: Minor cleanup after #6055 2019-06-17 00:31:38 +03:00
Megamouse 5928dd826c Qt: add Remove All Caches to the game context menu 2019-06-16 23:07:13 +03:00
Rui Pinheiro b94730fa95 Fix dynamic_library::loaded 2019-06-16 21:31:38 +03:00
Megamouse b29a89807e cellPad: set len to 0 if pads are being intercepted by the system 2019-06-15 00:24:10 +02:00
Megamouse 3f00b485a0 cellMsgDialogAbort: don't call on_close and properly re-enable pads 2019-06-15 00:24:10 +02:00
kd-11 8dafc8e542 cfg: Default MSAA level to auto 2019-06-14 17:19:55 +03:00
kd-11 e515d9b83a vk: Fixup for missing resource reference
- Missing ref increment when using framebuffer could lead to use-after-free.
  How master was not crashing is surprising
2019-06-14 16:19:52 +03:00
kd-11 c90186cf35 vk: Do not use pixel_center_origin as its use is explicitly restricted by spec 2019-06-14 16:19:52 +03:00
kd-11 98156d2a2c vk: Avoid submitting wrong sample count in overlay passes 2019-06-14 16:19:52 +03:00
kd-11 4104d7a6a1 vk: Simplify WCB heuristics and fix out-of-bounds access 2019-06-14 16:19:52 +03:00
kd-11 86119f58d6 rsx: Typo fix 2019-06-14 16:19:52 +03:00
kd-11 9d166c5bed rsx: Force invalidate of children by issuing a resolve notification whenever the parent is written to
- Fixes successive reads of an antialiased surface that is still bound between reads
2019-06-14 16:19:52 +03:00
kd-11 296e0105c4 vk: Fix WCB for antialiased memory 2019-06-14 16:19:52 +03:00
kd-11 9d0f5aedf3 vk: Add some missing barriers 2019-06-14 16:19:52 +03:00
kd-11 e4671c29a6 rsx: Fix typo
- Arguments to the transform function are xxyy not xyxy
2019-06-14 16:19:52 +03:00
kd-11 8a1cf2c913 rsx: Attempt to reduce stencil load overhead for nvidia cards 2019-06-14 16:19:52 +03:00
kd-11 ca82dd7200 vk: Improve overlay passes for resolve/unreolve
- Refactor overlays and resolve passes to support use of push constants instead of relying buffer map/unmap
- Add support for nvidia resolve (NV is the only vendor not supporting shader_stencil_export)
2019-06-14 16:19:52 +03:00
kd-11 c655036920 rsx/fp: Ease pressure on fragment shaders when emulating clamp16
- TODO: Option to completely skip clamping in some architectures as it is not needed in most games
- Mostly affects older GPUs that do not have access to native fp16
2019-06-14 16:19:52 +03:00
kd-11 5f34c0c59a vk: Clean up WCB readbacks when resource is multisampled
- Resolve image first before performing any transfer operations
2019-06-14 16:19:52 +03:00
kd-11 9d314ca4ca rsx: Correctly count number of valid entries if there are broken entries in the cache 2019-06-14 16:19:52 +03:00
kd-11 bca5f94b3f rsx: Add option to toggle MSAA 2019-06-14 16:19:52 +03:00
kd-11 ea8409dcfd rsx: Re-enable optional sample-to-pixel transformation 2019-06-14 16:19:52 +03:00
kd-11 acb14320da rsx: Fixup for resolution scaling support 2019-06-14 16:19:52 +03:00
kd-11 4a5bbba277 rsx: Enable MSAA
- vk: Enable depth buffer resolve+unresolve
- vk: Add AMD stenciling extension support
- rsx: Temporarily disables MSAA-compatible hacks such as transparency AA
- TODO: Add paths to optionally disable MSAA
2019-06-14 16:19:52 +03:00
kd-11 f6f3b40ecc rsx: Fix AA coordinate transforms
- Requires native_pitch value to take samples into account
2019-06-14 16:19:52 +03:00
kd-11 655eff29e8 rsx: Refactoring and cleanup after d3d12 separation
- Remove deprecated functionality
- Refactor to share code between common routines
2019-06-14 16:19:52 +03:00
kd-11 db5d56a22d d3d12: Remove all shared code with other backends 2019-06-14 16:19:52 +03:00
kd-11 0d906d6974 rsx: Remove surface aa_mode hacks 2019-06-14 16:19:52 +03:00
Megamouse ac3982d2b0 Input: fix evdev buttons 2019-06-13 19:12:46 +02:00
scribam 595ad15499 Add FUNDING.yml 2019-06-12 18:20:52 +03:00
scribam 13671d9684 rsx: Apply Clang-Tidy fix "modernize-loop-convert" + const when relevant 2019-06-12 15:11:52 +03:00
scribam 1e327ad31b rsx: Apply Clang-Tidy fix "readability-avoid-const-params-in-decls" 2019-06-12 15:11:52 +03:00
scribam 370dcd9d6e rsx: Apply Clang-Tidy fix "readability-simplify-subscript-expr" 2019-06-12 15:11:52 +03:00
scribam 0b97d12a7b rsx: Apply Clang-Tidy fix "modernize-use-using" 2019-06-12 15:11:52 +03:00
scribam f1e939936a rsx: Apply Clang-Tidy fix "modernize-use-override" 2019-06-12 15:11:52 +03:00
scribam 44265aa27d rsx: Apply Clang-Tidy fix "modernize-use-equals-default" 2019-06-12 15:11:52 +03:00
scribam 635695ac78 rsx: Apply Clang-Tidy fix "modernize-use-emplace" 2019-06-12 15:11:52 +03:00
scribam a555504142 rsx: Apply Clang-Tidy fix "modernize-deprecated-headers" 2019-06-12 15:11:52 +03:00
scribam cba828384d rsx: Apply Clang-Tidy fix "modernize-pass-by-value" 2019-06-12 15:11:52 +03:00
scribam a02a8642b0 rsx: Apply Clang-Tidy fix "modernize-make-unique" 2019-06-12 15:11:52 +03:00
scribam b91bcdbbca rsx: Apply Clang-Tidy fix "modernize-use-bool-literals" 2019-06-12 15:11:52 +03:00
scribam 349e7c8708 rsx: Apply Clang-Tidy fix "readability-non-const-parameter" 2019-06-12 15:11:52 +03:00
scribam 35dc98be06 rsx: Apply Clang-Tidy fix "readability-string-compare" 2019-06-12 15:11:52 +03:00
scribam ac7e89660f rsx: Apply Clang-Tidy fix "readability-redundant-smartptr-get" 2019-06-12 15:11:52 +03:00
scribam 801fa0113f rsx: Apply Clang-Tidy fix "readability-inconsistent-declaration-parameter-name" 2019-06-12 15:11:52 +03:00
scribam c9b0a4afd0 rsx: Apply Clang-Tidy fix "performance-type-promotion-in-math-fn" 2019-06-12 15:11:52 +03:00
scribam 8f2647555a rsx: Apply Clang-Tidy fix "readability-redundant-string-init" 2019-06-12 15:11:52 +03:00
scribam 331fe01762 rsx: Apply Clang-Tidy fix "performance-for-range-copy" 2019-06-12 15:11:52 +03:00
scribam db926ee671 rsx: Apply Clang-Tidy fix "performance-unnecessary-value-param" 2019-06-12 15:11:52 +03:00
scribam 81a3b49c2f rsx: Apply Clang-Tidy fix "readability-container-size-empty" 2019-06-12 15:11:52 +03:00
scribam c4667133c4 gl/vk: Add constexpr to varying_registers and sync functions between the two backends 2019-06-12 10:59:31 +01:00
scribam 497f0c26e7 utilities: Add constexpr to color4_base
For consistency with the others colorX_base
2019-06-12 10:59:31 +01:00
scribam 65581acbf9 rsx: Use constexpr for flattening_helper::m_register_properties 2019-06-12 10:59:31 +01:00
Eladash d6db61c594 misc atomics: Use std::atomic_thread_fence (MSVC) 2019-06-11 12:18:04 +03:00
scribam b3513cd7fa Set minimum supported version of GCC to 8.x
And update the documentation accordingly
2019-06-11 00:48:11 +03:00
kd-11 d361eedbec rsx: Clean up window management code
- Removes a lot of wm_event code that was used to perform window management and is no longer needed.
- Significantly simplifies the vulkan code.
- Implements resource management when vulkan window is minimized to allow resources to be freed.
2019-06-10 14:57:03 +03:00
kd-11 57196f0504 vk: Move frame present synchronization to the driver
- Just use a semaphore and let the driver handle it instead of manual framepacing.
  We lose framepace control but drivers have matured in the past few years so it should work fine.
2019-06-10 14:57:03 +03:00
Talkashie 527f7a437a important fix (#6067)
Fix typo in Pad Settings Dialog
2019-06-10 13:03:55 +03:00
RainbowCookie32 652e32341d Update DS3 message in Pads settings 2019-06-10 00:52:22 +01:00
scribam 8646e90832 3rdparty: Update stb_image.h and stb_truetype.h 2019-06-09 23:43:57 +01:00
scribam b190e1f50b 3rdparty/vk: Update VulkanMemoryAllocator to version v2.2.0 2019-06-09 23:43:57 +01:00
scribam 39fa1d7031 ci/vk: Bump Vulkan version (1.1.73.0/1.1.97.0 => 1.1.106.0)
VULKAN_SDK_MIRROR removed as the server is down
2019-06-09 23:43:57 +01:00
scribam f9ad635856 rsx: TextGlyphs optimizations 2019-06-09 23:09:11 +01:00
Nekotekina 2bc0ea37ab sys_fs: yield PPU on disk access ops 2019-06-08 19:34:55 +03:00
Megamouse e8a516529b fix games.yml 2019-06-08 15:47:11 +03:00
Nekotekina 1dedeb37bf Fix lf_queue_base::wait
Add missing state change
2019-06-05 15:51:25 +03:00
Nekotekina b2e08100ff Fix sad bug 2019-06-04 19:13:30 +03:00
Nekotekina 9dc0368079 Rename cond_x16 to shared_cond
Extend capacity from 16 to 32.
Remove redundant m_total counter.
2019-06-04 16:37:50 +03:00
Nekotekina 447029a700 Implement fs::file::write_gather (Vectored I/O) 2019-06-03 23:18:39 +03:00
Nekotekina 31994dd3b2 Rename cond_one to unique_cond
Remove redundant in_cv in cellVdec
2019-06-02 23:22:16 +03:00
Nekotekina 0333942795 Change utils::popcnt16 to popcnt32
Use __builtin_popcount if available.
2019-06-02 23:22:16 +03:00
Nekotekina dfd50d0185 Implement std::bit_cast<>
Partial implementation of std::bit_cast from C++20.
Also fix most strict-aliasing rule break warnings (gcc).
2019-06-02 23:22:16 +03:00
scribam 790962425c Fix some "-Wpedantic" warnings 2019-06-01 22:59:23 +03:00
scribam 99c1f87289 vk: Fix memory value in comments to match with the code below 2019-06-01 22:59:23 +03:00
scribam 66946a83e6 keyboard: Replace integer literals by bool when relevant
Recommendation from Clang-Tidy: https://clang.llvm.org/extra/clang-tidy/checks/modernize-use-bool-literals.html
2019-06-01 22:59:23 +03:00
scribam 8a6b5ca71f Access class static methods with "::"
Recommendation from Clang-Tidy: https://clang.llvm.org/extra/clang-tidy/checks/readability-static-accessed-through-instance.html
2019-06-01 22:59:23 +03:00
scribam 09c9996f31 Use empty() instead of comparing size() with 0
Recommendation from Clang-Tidy: https://clang.llvm.org/extra/clang-tidy/checks/readability-container-size-empty.html
2019-06-01 22:59:23 +03:00
scribam bf557ea6e6 Use the more efficient character literal overload for find_first_of/find_last_of
Recommendation from Clang-Tidy: https://clang.llvm.org/extra/clang-tidy/checks/performance-faster-string-find.html
2019-06-01 22:59:23 +03:00
scribam 78c7ef3039 rsx: Use clear() instead of resize(0)
The result is the same but clear [1] has slightly less code than resize [2] and signals better the intent IMHO.

[1] https://github.com/gcc-mirror/gcc/blob/fb7fb646fa1631f429dcca7201f4a1a77f876667/libstdc%2B%2B-v3/include/bits/stl_vector.h#L1495
[2] https://github.com/gcc-mirror/gcc/blob/fb7fb646fa1631f429dcca7201f4a1a77f876667/libstdc%2B%2B-v3/include/bits/stl_vector.h#L934
2019-06-01 22:59:23 +03:00
Malcolm Jestadt f90f68d129 Detect tsx_force_abort presence and print to log 2019-06-01 17:46:57 +03:00
Megamouse cafa4521ee Better Support For Game Collections (#4450) 2019-06-01 01:12:30 +03:00
scribam 141d78344a cmake: do not build glslang SPVRemapper 2019-06-01 00:01:32 +03:00
scribam d9d7634f8b vk: remove duplicate condition in pipeline_props struct equal operator 2019-06-01 00:01:32 +03:00
msuih 32776db8a6 Remove SPU verification from settings 2019-05-31 19:49:32 +03:00
msuih ef587d4cdc Limit shaderlog writing behind log_programs setting 2019-05-31 19:49:32 +03:00
kd-11 f2cac26154 rsx: Refactor out GLSLTypes from GLSLCommon to avoid warning spam due to unused functions when included in settings dialog code 2019-05-31 13:27:43 +03:00
kd-11 6e92516070 vk: Do not reset descriptors from the aux buffer when things are running slow
- The aux buffer borrows its descriptors from the lagging frame, so they are still in use until the frame completes.
2019-05-31 13:27:43 +03:00
kd-11 e118c9e5da update glslang 2019-05-30 11:48:38 +03:00
Megamouse c638e82678 Qt: fix a crash when closing rpcs3 after downloading the compat database 2019-05-28 21:47:49 +02:00
Megamouse fb1d8cb548 Qt/Input: keep LED colors when setting vibration and merge pad functions 2019-05-28 21:47:49 +02:00
Megamouse 5760aa2e57 Qt: Use correct LED color in pad settings dialog 2019-05-28 21:47:49 +02:00
Megamouse e4fca8274e PSF: Don't attempt to load files with size 0 2019-05-28 21:47:49 +02:00
Megamouse f48589005d handle some more warnings 2019-05-28 21:47:49 +02:00
Megamouse 8ebe02ce32 Qt: properly disable the "Filter Noise" button in pad_settings_dialog 2019-05-28 21:47:49 +02:00
Megamouse 3a5d1c6b15 Input: Add simple stick multipliers 2019-05-28 21:47:49 +02:00
Megamouse 34964e0e4f handle some warnings 2019-05-28 21:47:49 +02:00
kd-11 d9ab2c7104 vk: Bump shaders cache version
- Pipeline properties changed with the renderpass update
2019-05-28 15:28:30 +03:00
kd-11 57eb892153 vk: Refactor framebuffers
- Refactor out framebuffers from the renderer core
- Use a proper cache with sorted queues for faster searching
2019-05-28 15:28:30 +03:00
Nekotekina 71b71537a0 SPU TSX: implement Accurate PUTLLC option
Allow spurious PUTLLC failure if disabled (default).
2019-05-25 22:23:23 +03:00
Nekotekina b839cc9d5b SPU TSX: restore busy_wait in GETLLAR 2019-05-25 21:41:11 +03:00
kd-11 507ec8252b vk: Refactor renderpass management
- Ensures the current renderpass matches the image properties even when a cyclic reference is detected
- Solves SDK debug output error spam due to mismatching layouts and renderpasses
2019-05-25 14:07:29 +03:00
Malcolm Jestadt c348fec84b Warn AMD linux users about potential performance loss if not using RADV 2019-05-24 17:16:29 +03:00
Malcolm Jestadt 6ab3011eef vk: Check_window_status fixups
Intel ANV has been tested and verified to work without workaround
AMDVLK and the proprietary AMD driver have been confirmed to require workaround for window resizing
2019-05-24 17:16:29 +03:00
RipleyTom d616ce223c Official ds3 driver support 2019-05-22 16:56:40 +03:00
kd-11 370b9e196d vk: Improve descriptor pool management
- Add double-buffered descriptor pools to avoid use-after-free situations
- Make descriptor pools more configurable
- Also adds in a hack to allow renderdoc to capture properly
2019-05-22 01:18:46 +03:00
kd-11 46ba53f122 vk: Propagate more information to the driver
- Pass "correct" layout to descriptors
- TODO: Fix renderpass attachment descriptors which are inadvertently doing silent transitions
2019-05-22 01:18:46 +03:00
kd-11 c3b234f972 gl: Fix staging buffer size calculation 2019-05-22 01:18:46 +03:00
Malcolm 9a26c0abda Overlays: Fix timing 2019-05-21 13:01:38 +03:00
Nekotekina 7de3c410cf SPU/PPU: update reservation logic on TSX path transactions
Make use of lock bits in reservation counters.
On PPU, fallback to compare_and_swap instead of desperate retry.
On SPU, lighten write set on retry by 'locking' outside of the transaction.
2019-05-20 14:32:50 +03:00
Nekotekina 9abb303569 vm: expand reservation lock bit area to 7 bit
This is minor change.
2019-05-19 17:46:55 +03:00
Nekotekina ceaa669494 LLVM DSL: change array syntax again
I probably misunderstand something.
2019-05-19 17:46:55 +03:00
Nekotekina a6cde36dc4 Link LLVMAsmParser library 2019-05-19 17:46:55 +03:00
kd-11 8009e53642 rsx: Fix upload block range optimization
- The 'max' index should take the first assigned ID; fixes problems with divisors
2019-05-19 17:33:21 +03:00
kd-11 a245d9fb24 vk: DOuble general-purpose heap allocation to 128M and add a better diagnostic message for OOM 2019-05-19 17:33:21 +03:00
kd-11 0ef7b2aaff rsx: Use a saner model for swap queue handling
- Use a simple queue to avoid redundant checks over all the contexts
- Poll queue if RSX pipe is idle
- Only check the queue when the frame context is dirty (after a queue operation)
- Reset descriptors at the start of the frame context to avoid having to synchronize mid-frame
- Fully synchronize if a descriptor reset is required mid-frame (spec compliance; also fixes flickering verts on some hardware)
2019-05-19 17:33:21 +03:00
kd-11 dc749d3975 vk: Bump max number of allocated draw calls from 4k to 16k 2019-05-19 17:33:21 +03:00
Nekotekina 85920253b3 SPU ASMJIT: fix BISLED
Properly prevent losing branch target (addr) vs get_events result (flags).
Move get_events() to the beginning.
Match BISL indirect_branch() args.
2019-05-17 23:44:19 +03:00
Nekotekina 6416fee986 SPU ASMJIT: fix indirect branch target order
Check stack mirror first, as in SPU LLVM.
2019-05-17 23:44:19 +03:00
Nekotekina bf7ee2de4c SPU: fix spu_runtime::g_tail_escape (unused yet) 2019-05-17 23:44:19 +03:00
Nekotekina 2cf139c7a8 SPU analyser: minor Safe fix
Properly handle known indirect branch target as a function call target.
2019-05-17 23:44:19 +03:00
msuih 164a7636bf Add max SPURS threads to settings 2019-05-17 23:43:00 +03:00
kd-11 e3f68c66d8 rsx: Use a shared sampler pool instead of relying on the drivers 2019-05-17 22:51:40 +03:00
Megamouse edb1a32bb1 overlays: use L1 and R1 to step by 10 in the save data list 2019-05-17 20:21:23 +02:00
Megamouse 32bdd8ef7b overlays: move some code to cpp files 2019-05-17 20:21:23 +02:00
kd-11 4037225e98 vk: Workaround for cyclic feedback loops
- Transition attachments to LAYOUT_GENERAL in case of a feedback loop
  - Fixes appearance of garbage along polygon edges in some
post-processing passes.
  - Also reverse this transition when rendering goes back to normal
2019-05-17 16:41:17 +03:00
gidan80 9aa08313e3 Qt: Add custom pad configs
Add a custom pad config for every game.
2019-05-16 20:37:41 +02:00
kd-11 cb78522620 rsx: Fixup for uninitialized surface antialiasing mode 2019-05-16 19:25:26 +03:00
kd-11 45a13d0319 rsx: Fixup for lost aliased surfaces
- Intersection routines were changed and require explicit identification of the "old surface"
2019-05-16 19:25:26 +03:00
kd-11 05eb1e9193 rsx: Fix zombie image references from inside the texture cache
- Do not add locked orphans to the flush_always cache! They will not remove their cache entries as they are not bound
2019-05-16 19:25:26 +03:00
kd-11 214bb3ec87 rsx: Always initialize memory unless it is guaranteed to be wiped 2019-05-16 19:25:26 +03:00
kd-11 88290d9fab rsx: Hack around using data regions as transfer targets 2019-05-16 19:25:26 +03:00
kd-11 4182f9984d rsx: Propagate split section information back to the texture cache 2019-05-16 19:25:26 +03:00
kd-11 3c7d8a1099 rsx: Minor texture/surface scanning optimization
- Also re-enable optimization in blit engine accidentally disabled during debugging
2019-05-16 19:25:26 +03:00
kd-11 9f0090772a rsx: Fix write tagging when comments are transferred in by blit engine 2019-05-16 19:25:26 +03:00
kd-11 4b443be881 rsx: Fix self-intersection with previous occupant of the address being replaced 2019-05-16 19:25:26 +03:00
kd-11 b840f6da28 [WIP] rsx: Use a sane reference counting model 2019-05-16 19:25:26 +03:00
kd-11 e3cf3ab6b8 rsx: Minor fixes
- Fix transfer scaling (inverted)
- Fix under-estimated typeless acquisition when doing depth format scaling
2019-05-16 19:25:26 +03:00
kd-11 e02e27b2b3 rsx: Prevent out-of-bounds writes when resolving shader input textures
- The target area can also have padding!
2019-05-16 19:25:26 +03:00
kd-11 1c439f6198 vk: Fix some spec violations 2019-05-16 19:25:26 +03:00
kd-11 88c20afd3a rsx: Implement unaligned surface inheritance with hierachial contribution
- Allows render targets to behave like stacked 3D views same as shader inputs are resolved
- Basically implements most of 'Read Color/Depth Buffers" option for 'free'.
- Allows splitting RTV/DSV resources if they are superceded by a partial surface
- Also allows intersecting new resources through the surface cache for proper inheritance from other scattered data
- TODO: Refactor bind_surface_as_rtt and bind_surface_as_ds to reduce asinine code duplication
2019-05-16 19:25:26 +03:00
Nekotekina a016728a9e SPU LLVM: re-enable stack mirror
It was disabled due to other bugs which should be fixed now.
2019-05-16 04:12:08 +03:00
Nekotekina a69329fe02 SPU ASMJIT: Fix Giga mode
This is embarassing.
Fixed local instruction table computation (indirect branch).
2019-05-16 04:11:05 +03:00
Nekotekina 26d131ef29 SPU LLVM: Fix Giga mode
Forgot to adjust global chunk table computation.
2019-05-16 04:02:33 +03:00
Nekotekina 91897fa69d SPU LLVM/ASMJIT: fix BRA/BRASL instructions for PIC
Handle absolute branch addressing correctly.
2019-05-16 02:41:31 +03:00
Nekotekina f95ec8a37c SPU LLVM: simplify jump table computation
Remove one add operation and adjust constants instead.
2019-05-16 00:54:50 +03:00
Nekotekina b138d25b97 SPU Disasm: fix absolute addressing in some instructions.
STQA, LQA, BRA, BRASL instructions.
2019-05-16 00:53:37 +03:00
Nekotekina a921af1e96 SPU LLVM/ASMJIT: remove minor unnecessary code 2019-05-16 00:52:52 +03:00
Nekotekina 43ae4b3f33 SPU LLVM/ASMJIT: add missing PC clamping
Minor fix, since it's mostly impossible to overflow.
2019-05-16 00:51:47 +03:00
Nekotekina 007108100e SPU: implement spu_runtime::g_tail_escape
May help to avoid gateway costs in some cases.
2019-05-15 18:47:40 +03:00
Nekotekina 4e75d2c2f7 SPU LLVM: don't save $2 in optimized functions 2019-05-15 16:41:57 +03:00
Nekotekina adc7d96683 SPU LLVM: simplify function prototype
Pass only $3
2019-05-15 16:18:13 +03:00
Nekotekina 16401722f1 SPU LLVM: fix $SP passing in functions, write PC on halt
Allows to skip updating $SP in optimizable functions.
2019-05-15 15:42:03 +03:00
Nekotekina b2d0ca83fb LLVM DSL: simplify value_t template for array 2019-05-15 15:17:36 +03:00
Nekotekina 09eb633f69 SPU ASMJIT: increase stack frame size
It seems Windows has minimal stack frame size 0x28.
2019-05-15 02:16:08 +03:00
Nekotekina 3753d27aba SPU: fix Giga mode (kinda)
Don't scan before the entry point.
Disable stack mirror in SPU LLVM.
Improve analyser logic for holes.
2019-05-14 22:15:04 +03:00
Nekotekina c481472faf SPU ASMJIT: add PIC support (fix)
Also cleanup and adapt for GHC CC.
2019-05-14 22:15:04 +03:00
Nekotekina 82295d131a SPU LLVM: split LLVM IR dump to spu-ir.log
Also move disasm to spu_recompiler_base::dump.
Interleave disasm with block target info for convenience.
2019-05-14 22:15:04 +03:00
Nekotekina ea554ae828 Implement 'Max SPURS Threads' option (hack)
Pauses SPURS threads beyond limit automatically if set.
2019-05-14 22:15:04 +03:00
Nekotekina 1eed421774 SPU LLVM: use branch patchpoints again
Renewed and adapted for PIC and all branch types.
This may address performance degradation after #5923.
2019-05-14 22:15:04 +03:00
Nekotekina 2f6707d0a0 SPU LLVM: regain some efficiency
Avoid returns from the recompiler gateway, favoring tail calls.
This may address performance degradation after #5923.
2019-05-14 22:15:04 +03:00
Nekotekina f33b81545e SPU: implement recompiler gateway function in assembly
Use GHC calling convention directly for SPU object entry points.
This may address performance degradation after #5923.
2019-05-14 22:15:04 +03:00
Nekotekina a74fd27e3d SPU LLVM: fix SPU termination (spu_escape) on Windows
Adjust restored stack pointer for the lack of tail call.
2019-05-14 22:15:04 +03:00
Nekotekina cc8c635855 SPU: PIC support preview
SPU ASMJIT not supported yet.
Giga mode not supported properly.
2019-05-14 22:15:04 +03:00
z0z0z 7cf11c7637 Use setenv instead of qputenv
A user reported issues with Qt's hidpi scaling and RPCS3. They could not disable with their own environment variable because qputenv overwrites them, setenv with 0 as third arg doesn't.
2019-05-14 21:48:14 +03:00
drysalter 6b01c2f9f5 Further changes to Skyline and Envy 2019-05-14 09:16:45 +01:00
scribam 22f61caf9f GLTexture: add missing #pragma once directive 2019-05-12 18:32:11 +03:00
scribam 786dc6ef40 Use if constexpr in hash_struct function 2019-05-12 18:32:11 +03:00
scribam 6c5ea068c9 Remove redundant semicolons
Fix "-Wextra-semi" warnings
2019-05-12 18:32:11 +03:00
Rui Pinheiro 1f82a26a9c SPU LLVM: Fix Mega 2019-05-12 00:39:42 +03:00
scribam 3623f4343f gl/vk: clear scissor_setup_invalid bit along with scissor_config_state_dirty bit 2019-05-11 13:13:49 +03:00
Nekotekina 8194c92f1c SPU LLVM: disable GHC CC for chunks on Windows
Causes fatal error inside LLVM.
2019-05-11 02:35:16 +03:00
Nekotekina 5d33d9a3d9 Enable most warnings in GCC 2019-05-11 02:13:19 +03:00
Nekotekina 7492f335e9 SPU analyser: basic function detection in Giga mode
Misc: fix EH frame registration (LLVM, non-Windows).
Misc: constant-folding bitcast (cpu_translator).
Misc: add syntax for LLVM arrays (cpu_translator).
Misc: use function names for proper linkage (SPU LLVM).

Changed function search and verification in Giga mode.
Basic stack frame layout analysis.
Function detection in Giga mode.
Basic use of new information in SPU LLVM.
Fixed jump table compilation in SPU LLVM.
Disable broken optimization in Accurate xfloat mode.
Make compiled SPU modules position-independent in SPU LLVM.

Optimizations include but not limited to:
 * Compiling SPU functions as native functions when eligible
 * Avoiding register context write-out
 * Aligned stack assumption (CWD alike instruction)
2019-05-11 02:13:19 +03:00
Megamouse fce9d6a7b8 Qt/input: add LED color picker to pad settings dialog 2019-05-09 22:02:00 +02:00
eladash 7ead021aa7 rsx: Fix 3d swizzled texture to linear conversation 2019-05-08 23:48:39 +03:00
kd-11 feb77793df typo fix
- Minor argument change requested by Megamouse
2019-05-07 09:39:35 +03:00
Exfiltratior f1ff75d815 correct small mistake 2019-05-07 09:39:35 +03:00
eladash 13d8e33d9a Return ESRCH if ppu thread ID was not found in sys_cond_signal_to 2019-05-07 08:58:07 +03:00
eladash 4e2650af91 Fix sys_rwlock_wlock timedout event
If the rwlock is currently acquired by a writer signaling readers is wrong and will lead to EPERM for wunlock!
Only signal blocked readers if the rwlock is currently acquired by readers
2019-05-07 08:58:07 +03:00
eladash ca08418dc1 Fix sys_rwlock_runlock on waiting readers
readers can wait on the sleep queue if a writer lock has been blocked before it, in this case after runlock: writer should acquire the lock but the r's sleep queue is still not empty!
2019-05-07 08:58:07 +03:00
Megamouse 5141590729 overlays: add separate timestamp for the start of the d-pad interval 2019-05-06 22:00:40 +02:00
Malcolm Jestadt fd2bc95a7b overlays: Double dpad repeat rate 2019-05-06 22:00:40 +02:00
Megamouse 9aa9cbf1d8 travis: update mac build 2019-05-06 21:06:25 +02:00
Megamouse 87d493fe1b Qt: smoother custom config icons 2019-05-06 20:00:36 +02:00
Megamouse e883f4f880 Qt: center compat circles 2019-05-06 20:00:36 +02:00
Megamouse 767eaaa82a Qt: smoother game grid icons 2019-05-06 20:00:36 +02:00
Megamouse caa6895310 Qt: smoother game icons in list mode 2019-05-06 20:00:36 +02:00
Megamouse ea0f04e130 Qt: smoother compat circles in list mode 2019-05-06 20:00:36 +02:00
Megamouse f480ba13c0 Qt: show all unique game data entries 2019-05-05 16:29:50 +02:00
Megamouse c1e245ae73 Emu: msg_dialog_frame fixup: don't reject on Close to prevent Emu.Stop() 2019-05-05 16:29:50 +02:00
Megamouse b639584acc Emu/Qt: Fix Boot Recent when using BootGame(add_only=true) 2019-05-05 16:29:50 +02:00
Megamouse b0a24665e5 Emu: msg_dialog_frame fixes 2019-05-05 16:29:50 +02:00
Megamouse 8a4778ba8c Qt: add some batch operations 2019-05-05 16:29:50 +02:00
kd-11 9c346c92f3 gl: undo an accidental deletion 2019-05-05 13:37:55 +03:00
kd-11 2bec304cca vk: Allow some drivers to bypass window polling if not needed 2019-05-05 13:37:55 +03:00
drysalter f246a8666a Qt: Minor Visual Fixes (#5899)
* Increase minimum size of User Manager window so QPushButtons aren't squeezed together
* Add spacing to the links in the About dialog to match all other buttons in the GUI
* Make the links in the About dialog work on Linux
* Turn the tooltip on "Config" in toolbar into an imperative like the rest of them
2019-05-04 22:56:55 +02:00
Nekotekina a703460fc6 SPU ASMJIT: skip some unused analyser steps
May improve performance
2019-05-04 19:35:13 +03:00
Nekotekina ba1ec1d5d6 SPU analyser: remove use_ra from HBR
Since this is a hint instruction, we don't really use reg value here.
2019-05-04 18:33:58 +03:00
Nekotekina 45ce8db6cb SPU Analyser: fix reg origin regression
Propagate phi instead of claiming new values
2019-05-04 18:29:47 +03:00
Nekotekina 4bd022f778 SPU analyser: minor logic fix and cleanup
Don't fill any chunk info for now (design mistake).
2019-05-03 14:18:22 +03:00
Nekotekina 6c34d7104e SPU analyser: fix excessive workload list size
Typo grade; regression
2019-05-02 23:29:02 +03:00
Nekotekina 54dc617f39 SPU analyser: internal spu_itype optimization
Use only 1 byte for instruction type.
Flags are transformed into range comparisons.
2019-05-02 19:32:09 +03:00
Nekotekina 15bd3b8724 SPU: fix minor UB in STQD/LQD instructions 2019-05-02 18:00:49 +03:00
Nekotekina 2b4da18709 SPU LLVM: fix xfloat regression
It was an old bug with possible hidden use of deleted instructions.
2019-05-02 13:39:43 +03:00
Nekotekina d48dc29e55 SPU LLVM: fix perf regression
Bug in the analyser was created recently in #5882.
2019-05-02 13:39:43 +03:00
Nekotekina 69d2ea35b9 SPU: minor analyser cleanup 2019-05-02 13:39:43 +03:00
Nekotekina a4c4ee9cb2 SPU: fix excessive cache size regression 2019-05-02 13:39:43 +03:00
msuih bc42719c48 Put DS3/DS4 instructions to Wiki instead 2019-05-02 13:39:16 +03:00
kd-11 6b7cd458e3 rsx: Silence some diagnostics unless compiled with debugging options 2019-05-01 15:36:21 +03:00
kd-11 1d5c52f476 rsx: Ignore stencil clear flag if the stencil write mask is disabled 2019-05-01 15:36:21 +03:00
kd-11 48cb265c2c rsx: Bounds check on local resource for atlas merge.
- Local resources can also have padded pitch dimensions and false-positives on range overlap tests
2019-05-01 15:36:21 +03:00
kd-11 63f9b8e0c6 gl/vk: Minor cleanup 2019-05-01 15:36:21 +03:00
kd-11 ec9aa74008 rsx: Fix section base offset calculation for blit_dst targets which affects confirmed memory range
- Fixes flushes only writing partially to target memory
2019-05-01 15:36:21 +03:00
kd-11 4e3ec162e2 rsx: Fix broken texture cache search when flipping 2019-05-01 15:36:21 +03:00
kd-11 6feffe6ff6 rsx: Ignore transfer offsets when wrapping behaviour is expected 2019-05-01 15:36:21 +03:00
kd-11 f56a6548b0 gl: Remove workaround for AMD driver bug fixed in driver 19.4.3 2019-05-01 15:36:21 +03:00
Nekotekina 1bc5e27507 SPU LLVM: move reg origin search to analyser
Refactor SPU analyser (block_info struct).
Fill register use info (currently unused).
2019-05-01 00:37:15 +03:00
Nekotekina 1294e0d189 SPU LLVM: improve codegen in loops
Use a trick in check_state to improve LICM pass.
2019-05-01 00:37:15 +03:00
Nekotekina e09c6ea4b4 SPU analyser: add spu_iflag
Register information about register accesses.
2019-04-30 14:33:27 +03:00
Nekotekina 716737ecf2 LLVM DSL: expression matching (alpha)
Implement remaining instructions.
Implement match_expr method.
Implement helper methods.
2019-04-30 14:33:27 +03:00
Deep Sleep c24b74398f Update to README.md due to dependency
Add qtbase5-private-dev for Debian as dependency to the README.md since it is needed, probably due to qt5.11.3+.
2019-04-30 14:33:01 +03:00
RipleyTom 301dd82819 DS3 Linux fix 2019-04-30 00:31:34 +03:00
eladash 3bd29b8bac Fix Unregistered HLE function access 2019-04-29 23:04:16 +03:00
eladash ea1c9a2e17 Fix PPU Breakpoints and ppu_check_toc 2019-04-29 23:04:16 +03:00
drysalter 468e1e90c8 Touch-ups to Skyline and Envy (#5884)
* Touch-ups to Skyline and Envy

* Match disabled colors on sliders, checkboxes and comboboxes

* Align header titles with section contents

* Make slider grooves clickable, add opacity to tooltips

* Adjust opacity, clean some code

* Correct slider margins, add hover effect to game table on Nightfall
2019-04-29 20:02:50 +01:00
msuih be6d9af1ab Print OS info to log
Credit to @maximilian578 for help
2019-04-25 20:20:37 +03:00
kd-11 243df38360 rsx: Fix VP writes to CC with a MOV instruction
- When moving to CC, the operation has VEC flag disabled and also temp
regs disabled. Looks to be the catch-all ELSE in the selection logic.
2019-04-25 16:23:05 +03:00
kd-11 3cbccdd760 rsx: Fragment shader decompiler cleanup
TODO: Investigate the _s input modifier behaviour further, in case it can avoid generating zeroes from a MAD instruction.
x = MAD(+ve, -ve, -ve) with _s input modifier in BFBC expects result to be Non-zero
2019-04-25 16:23:05 +03:00
kd-11 4cd1c25729 "rsx: Ignore argument sign for SQRT operations" 2019-04-25 16:23:05 +03:00
kd-11 32396ba366 rsx: Simplify use of some mixed input functions using OPFLAGS to avoid implicit conversions 2019-04-25 16:23:05 +03:00
kd-11 f12bd8068c rsx: Fragment decompiler fixups
- Properly test for NaN and Inf when clamping down to fp16
- Optimize divsq a bit; mix(vec, vec, bvec) emits OpSelect which is what
we want here, instead of component-wise selection which is much slower.
2019-04-25 16:23:05 +03:00
kd-11 abe7188acf rsx: Proper workaround for broken DIVSQ instruction on realhw
- While mul(0, nan) = nan and 0 / 0 = nan, 0 / sqrt(0) = 0 because of hw
gremlins. normalize(0) is also nan so this behaviour does not work
around that particular case either which makes it even more baffling.
2019-04-25 16:23:05 +03:00
kd-11 60f3059d22 rsx: Compensate for nvidia's low precision attribute interpolation
- The hw generates inaccurate values when doing perspective-correct
  interpolation of vertex output attributes and makes the comparison (a ==
  b) fail even when they are a fixed constant value.
- Increase equality tolerance when doing comparisons in fragment
  shaders for NV cards only to work around this issue.
- Teepo fix
2019-04-25 16:23:05 +03:00
kd-11 463b1b220d rsx: Improve accuracy of shadow compare Ops when non-integer depth formats are used
- The fixed-point D24S8 format does special Z clamping during compare which matches PS3 behaviour
- D32S8 is a floating point format and comparison with Dref > 1 always fails causing black edges/borders
2019-04-25 16:23:05 +03:00
kd-11 7ad1646c2c vk: Skip feature check if extension is not supported 2019-04-25 16:23:05 +03:00
kd-11 06a85f00d1 rsx: Shader decompiler cleanup and improvements
- Improve support for float16_t by minimizing mixed inputs to functions
(ambiguous overloads)
- Minimize amount of downcasts in code by using opcode flags
- Re-enable float16_t support for vulkan
2019-04-25 16:23:05 +03:00
kd-11 a668560c68 rsx: Use native half float types if available
- Emulating f16 with f32 is not ideal and requires a lot of value clamping
- Using native data type can significantly improve performance and accuracy
- With openGL, check for the compatible extensions NV_gpu_shader5 and
AMD_gpu_shader_half_float
- With Vulkan, enable this functionality in the deviceFeatures if
applicable. (VK_KHR_shader_float16_int8 extension)
- Temporarily disable hw fp16 for vulkan
2019-04-25 16:23:05 +03:00
kd-11 ee319f7c13 rsx: Implement strict clamp16 operation needed for NVIDIA cards 2019-04-25 16:23:05 +03:00
Nekotekina 2ade3c594c LLVM DSL: expression matching (preview 2)
Implement more instructions.
2019-04-25 03:33:18 +03:00
Nekotekina aca61fdcf9 LLVM DSL: implement expression matching (preview)
Only literal match for binary ops implemented.
2019-04-24 23:55:41 +03:00
Nekotekina 8754bbd444 SPU LLVM: add match_vr<> template
Returns reg value only if type is compatible, avoiding bitcast.
2019-04-24 23:55:41 +03:00
Nekotekina dd9bd1338b SPU LLVM: add get_vrs<> template 2019-04-24 23:55:41 +03:00
Nekotekina 3e0b45719d LLVM DSL: rewrite zshuffle, shuffle2, build
Add llvm_const_vector template.
2019-04-24 23:55:41 +03:00
Nekotekina b02503963e LLVM DSL: rewrite splat, fsplat, vsplat
Add llvm_const_float and llvm_splat templates.
2019-04-24 23:55:41 +03:00
Nekotekina c83e65f29e LLVM DSL: rewrite extract and insert 2019-04-24 23:55:41 +03:00
Nekotekina b7b93eae13 SPU LLVM: minor bitcast cleanup
Remove redundant explicit constand propagation in some instructions.
2019-04-24 23:55:41 +03:00
Nekotekina 2eac59f59a LLVM DSL: rewrite add_sat and sub_sat
Simplify constant folding logic
2019-04-24 23:55:41 +03:00
Nekotekina ac473eb400 Rewrite cpu_translator::rol, add fshl and fshr
Use new funnel shift intrinsics
2019-04-24 23:55:41 +03:00
Nekotekina 42448cf3e5 Remove cpu_translator::scarry, cpu_translator::merge 2019-04-24 23:55:41 +03:00
Nekotekina 524aac75ed LLVM DSL: rewrite bitcast, zext, sext, trunc, select, min, max ops
Are made composable in expressions similar to arithmetic ops.
Implement noncast in addition to bitcast (no-op case).
Implement bitcast constant folding.
Fixed some misuse of sext<>.
2019-04-24 23:55:41 +03:00
Nekotekina 3925cb59ac LLVM DSL: fix pointer type traits
Clear and match 'void' type
2019-04-24 23:55:41 +03:00
Nekotekina dc9118ef50 LLVM DSL refactoring
Properly forward value categories in expression structs.
Simplify SFINAE tests (is_llvm_expr, llvm_common_t) in global operators.
Add llvm_const_int and remove llvm_add_const, llvm_sub_const, etc.
Add llvm_ord and llvm_uno for FP comparison via >=< operators.
Replace cpu_translator::fcmp with fcmp_ord and fcmp_uno.
2019-04-24 23:55:41 +03:00
Megamouse 467ef2afca Qt: also spawn exit game dialog if rpcs3 is closed (e.g. by accident) 2019-04-24 21:48:36 +02:00
Megamouse cb5dd1353b Qt: force always on top to fix dialogs appearing behind the game window 2019-04-24 21:48:36 +02:00
Megamouse bb0609372d Qt: go out of fullsceen if exit game pop up is triggered 2019-04-24 21:48:36 +02:00
drysalter 258df80bf6 Add Envy and Skyline themes (#5789) 2019-04-21 00:41:20 +01:00
Ani a24ede4f40 cellPad: Update vendor and product IDs
- Used IDs were not from the Guitar Hero instruments but in fact from the Rock Band ones. Sets the correct Guitar Hero IDs and adds the Rock Band ones on comments. TODO: Allow selecting the specific devices on the PAD Settings.
- Adds DJ Hero Turntable VID/PID.
- Adds Dance Dance Revolution Mat VID/PID.
2019-04-20 23:17:13 +01:00
eladash 3a5f4ed757 Print SPU Group ID on the debugger 2019-04-20 20:43:58 +01:00
eladash 450e2c9a0e cellSaveData: Add missing SDK version check for setParam->reserved2 check 2019-04-20 20:43:58 +01:00
eladash ae5a4b697e Fix cellSaveDataListAutoSave/Load unk flags 2019-04-20 01:04:41 +03:00
eladash 1e9d3346d1 Reschedule in cellMsgDialogOpen2 2019-04-20 01:04:41 +03:00
eladash 9446bd2d3f Handle a few more cellSaveData errors
* Check directory existence if setParam is NULL (dont create directory)
* Fix mask for reCreateMode
* Check a few setParam fields including reserved buffers.
* Fix sizeKb when the dir is empty except from PARAM.SFO
* Fix error checking when CELL_SAVEDATA_RECREATE_YES is specified but setParam is NULL (Doesnt do anything, simply errors)
2019-04-20 01:04:41 +03:00
eladash 6f76e34104 rsx: Fix race on clearing native_ui vs emu_requested flag 2019-04-20 01:04:41 +03:00
eladash 67f098627a Fix sys_spu group ID 2019-04-20 01:04:41 +03:00
eladash ff11d9a3bd Improve scheduler control for cellSaveData
TODO: There are probably more spots where we should yield.
A little more at the start because PacketRead is called twice.
Dont use sys_timer_usleep because it will just call this_thread::yield() repeatedly.
2019-04-20 01:04:41 +03:00
eladash 9497270da5 Implement initial arguments error checking for cellSaveData 2019-04-20 01:04:41 +03:00
eladash 2b4bc588dc Put missing check_state() in some places
Fixes a few verification failures while closing the emulator with HLE liblv2
2019-04-20 01:04:41 +03:00
eladash 888cb9d673 Remove reader_lock executed in every instruction by RSX
Use optimistic double check instead, use one load instruction for the check to be atomic

+ Read emu status once every FIFO iteration
2019-04-20 01:04:41 +03:00
eladash f25587d24c rsx: Write vblank semahpre, minor semaphore acquire optimization 2019-04-20 01:04:41 +03:00
eladash 777a99d01b misc: Lower default perf overlay detail
Because RSX Guest utilization confuses people and is only meaningful for debugging.
2019-04-18 22:23:05 +03:00
Nekotekina 7865982208 Fix static_hle log channel definition 2019-04-16 23:49:18 +03:00
Nekotekina 52c589ed3d Revert disabling AVX path in SPU verification.
It was experimental and builds for tests are available in history.
2019-04-16 23:49:18 +03:00
Nekotekina 9060177dbd SPU transactions: add SSE path if AVX is not available
This handles hypothetical situation when AVX is disabled system-wise.
Also refactored register use, to match Windows path with Linux path.
This reduces read set a little at the cost of stack use.
2019-04-16 23:49:18 +03:00
Megamouse 1152317b47 libusb VS stuff 2019-04-16 22:47:24 +02:00
Megamouse b929c13c45 implement get_firmware_version
add firmware version to the first line in the log
2019-04-16 22:13:28 +02:00
Megamouse 092cfef060 Qt: add more filters to firmware installation file dialogs 2019-04-16 21:39:14 +02:00
msuih 8f333117eb Set stack limit properly 2019-04-16 18:39:57 +03:00
msuih baf42430d6 Decrease severity of sys_net_bnet_close 2019-04-16 18:39:57 +03:00
Jordy Vieira f995771143 Ignore sorting on ranged combobox and log message 2019-04-14 19:15:23 +03:00
Jordy Vieira bad0415f05 Change EnhanceComboBox to optionally sort the settings list before adding it to the combo box 2019-04-14 19:15:23 +03:00
Nekotekina 136fc8cfe3 SPU ASMJIT: avoid AVX in verification (experimental) 2019-04-14 18:03:45 +03:00
Nekotekina b97a1aeca2 Increase max stack size to 8 MB on Windows
Default is 1 MB and it could cause problems in compiling heavy SPU LLVM functions.
2019-04-14 18:03:45 +03:00
Nekotekina c1edae73c5 SPU ASMJIT: move vzeroupper a bit 2019-04-14 15:10:54 +03:00
Nekotekina 8deb20e928 SPU: write cache before compiling 2019-04-13 22:56:11 +03:00
Nekotekina e064b92058 mutex.h: fix warning (unary minus unsigned) 2019-04-13 22:56:11 +03:00
Nekotekina d52df9352c fs::rename, use renameat2 on Linux 2019-04-13 22:56:11 +03:00
Nekotekina fe593de6d7 Update LLVM 2019-04-12 22:20:21 +03:00
kd-11 df3b46a611 rsx: Improve texture sourcing and clipping when reverse scanning is enabled
- When reverse scanning, offsets are inverted and offset value of 0 is logically equivalent to an offset of -1
- Add an explicit message if clipping happens to avoid silent errors/bugs
2019-04-12 15:36:21 +03:00
Nekotekina 0d415407c7 sys_fs_unlink: add CELL_EISDIR check 2019-04-12 12:24:36 +03:00
Nekotekina f40320bcae Fix cellVdecOpen
Use pseudo-address in sys_ppu_thread_create calls
2019-04-11 21:20:22 +03:00
Nekotekina c34821ded2 Travis: disable gcc build
Temporary optimization
2019-04-11 18:57:56 +03:00
TGEnigma 38cc92ec45 Add _sys_ppu_thread_create and sys_ppu_thread_rename error checks 2019-04-11 18:14:05 +03:00
eladash 8da78c098c SPU LLVM: Fix branch to self at start of block state check 2019-04-11 17:47:52 +03:00
eladash eba8e2284b SPU LLVM: Fix CFLTU
Clamp properly result from both sides!

TODO: Figure out whats different CreateFPToUi has from CFLTU and why it fails here.
2019-04-11 17:47:52 +03:00
eladash 969af86eba SPU: Implement BISLED
DFCMGT instruction removed, it was wrong to add to begin with

ASMJIT: Fix compilation of double compare instructions, move exception to runtime instead of compiletime!
Jarves confirmed that he implemented this instruction because of that bug with asmjit only, affected God Of War 3
2019-04-11 17:47:52 +03:00
eladash b307aff9eb Prefetch byteswapped opcodes in ppu interpreter 2019-04-11 17:47:52 +03:00
eladash 1c462abc37 Make sure to update cia when calling in unknown hle table func access 2019-04-11 17:47:52 +03:00
eladash 3304e3b0b7 PPU LLVM: Fix STSWI and LSWI 2019-04-11 17:47:52 +03:00
eladash f028737db8 Implement fallback for PPU LLVM
This matches with interpreter implementation, fixing unregistered functions in lost cases
2019-04-11 17:47:52 +03:00
eladash a9014a8cac ppu Fast/Precise: Fix SIMD instructions VSUM2SWS, VPKSWSS, VPKSHUS, VPKSHSS
Also rewrite VPKSHUS for speed.
2019-04-11 17:47:52 +03:00
eladash e21504d52d ppu interpreter: Improve FPCC field handling 2019-04-11 17:47:52 +03:00
eladash aa44ef1f44 Fix default PPU nj status
TODO: Support  it...
2019-04-11 17:47:52 +03:00
eladash d555eeb0f4 Check start status in sys_prx_start/stop_module 2019-04-11 17:47:52 +03:00
msuih af86d1d77a Fix timespec for MSVC 2019-04-11 16:29:48 +03:00
Inviuz 52a12185a0 Initial sys_overlay 2019-04-10 23:25:09 +03:00
scribam 1d947daa81 hle: Add some more functions 2019-04-10 22:15:35 +03:00
Megamouse ec2d882199 Qt: add version update hint to game list 2019-04-10 15:37:37 +01:00
Nekotekina 40142420c1 Implement vfs::host::unlink
Emulate POSIX behaviour in sys_fs_unlink.
This should allow to delete opened files transparently on Windows.
2019-04-10 13:58:12 +03:00
Nekotekina 0736fc8b28 Add fs::file::set_delete 2019-04-10 13:58:12 +03:00
Nekotekina a54d9c527f Simplify fs::file::get_handle
Via fs::file_base::get_native_handle method
2019-04-10 13:58:12 +03:00
Nekotekina 9736773c04 Implement vfs::host::rename
With spurious access error workaround
2019-04-10 13:58:12 +03:00
Nekotekina 3354f068fc PPU/SPU transactions: ease cache line interference (TSX path)
Touch memory on the same memory page, but different cache lines.
2019-04-10 13:58:12 +03:00
kd-11 12dc3c1872 vk: Dynamic heap management to potentially fix ring buffer overflows
- Allows checking one heap type at a time, on demand
- Should avoid OOM situations unless inside an uninterruptible block
2019-04-09 13:40:54 +03:00
kd-11 a4495c35b7 rsx: Fixups for swizzled texture scanning
- Revert to using block metrics, but with optional per-channel decode
stage for the final transfer. Much cleaner than hacking in the width to
be in channels instead of blocks.
2019-04-09 13:40:54 +03:00
kd-11 a5ed30a8c0 rsx: Fixups for data cast operations via typeless transfer 2019-04-09 13:40:54 +03:00
kd-11 f04a0a2bb6 rsx: Remove some old restrictions affecting memory persistence 2019-04-09 13:40:54 +03:00
kd-11 0a604e39f1 rsx: Implement RGB655 decode 2019-04-09 13:40:54 +03:00
kd-11 cc3809fbfe gl: Register a few more missing formats for conversion 2019-04-09 13:40:54 +03:00
kd-11 e4e86455f2 rsx: Fix temporary subresource caching behaviour
- Do not cache if a gathered subresource contains a bound RTT
- Change op to dynamic copy if parent is still bound
2019-04-09 13:40:54 +03:00
kd-11 3249000511 rsx: Improvements to texture scanning
- Removes CPU-only transforms that broke GPU-side code.
 -- Channels in GPU compute are laid out in cell-order, but CPU was uploading in favorable order and compensating with swizzles.
 -- This leads to 2 different layouts depending on the location of the data (CPU vs GPU)
- Implement R8G8_R8B8 interleaved format decode
- General improvements
2019-04-09 13:40:54 +03:00
kd-11 0f7af391d7 vk: Implement copy-to-buffer and copy-from-buffer for depth_stencil
formats
- Allows D24S8 and D32S8 transport via typeless channels
- Allows uploading and downloading D24S8 data easily
- TODO: Implement optional byteswapping to fix flushed readbacks with
the same method
2019-04-09 13:40:54 +03:00
kd-11 366e4c2422 rsx: Preliminary support for format conversions using typeless resolve 2019-04-09 13:40:54 +03:00
kd-11 b7470cfc1a rsx: Tighten format checks in cache hit tests 2019-04-09 13:40:54 +03:00
kd-11 443fde760f rsx: Blit engine clipping fixes
- Do not round up sub-pixel offsets, round down instead
- Do not allow incomplete sources for hw blit transfer
- Reimplement src clipping (slice_h)
- Check 'area' of incoming texels and correct for them before RTT lookup/transfer
- Filter out incomplete targets when performing RTT lookup (1 texel or less contribution)
2019-04-09 13:40:54 +03:00
scribam f30af3ccd2 hle: Add more missing functions 2019-04-07 23:31:15 +03:00
JohnHolmesII 7625f7df11 Correct travis deploy behavior (bug).
cd returns an error code, thus overwriting '0'
2019-04-06 16:52:31 +03:00
scribam 8dbf2638e2 hle: Add some missing functions
0xBA50BC23 => cellCelpEncOpenExt
0x1AC58D11 => cellHttpFlushCache
0xA39FE9DC => cellHttpEndCache
0xB4FA3111 => cellHttpInitCache
0x4A18A89E => sceNpMatchingSetRoomInfoNoLimit
0xB020684E => sceNpMatchingGetRoomInfoNoLimit
2019-04-05 17:28:10 +01:00
Alex James d7ad991b7e Fix macOS compilation
This is needed for GL/glew.h to be found.
2019-04-02 03:16:55 +03:00
Violette Jansson 3bfe921a7a updated domain name for vulkan mirror
obk.ee expired the new domain is nkmk.ch, renewing a .ee domain is expensive and annoying so i moved to a new tld.
this .ch domain will not expire for around 3 years normally.
2019-04-01 13:46:32 +03:00
Jocker666z 2119566da7 add qtdeclarative5-dev fo debian/ubuntu
if not installed, cmake fail on -> '/usr/lib/x86_64-linux-gnu/cmake/Qt5Qml/Qt5QmlConfig.cmake'
2019-03-31 22:43:00 +03:00
eladash 182054b8af Implement sys_vm_append/return_memory 2019-03-31 14:57:21 +03:00
eladash 3c0564c9b7 Fix timer state after event queue was destroyed
* Hw tests show state is unaffected by external destruction of the event queue

* Minor race regarding state check fixed (can result in an undestroyable state)
2019-03-31 14:57:21 +03:00
eladash 90490f775d Fix sys_timer_usleep specifically with 0 sleep time
Remove context switch, replace it with host yield() for giving some cpu time for SPUs ans RSX
2019-03-31 14:57:21 +03:00
eladash 8185ef7610 rsx: Improve vblank accuracy 2019-03-31 14:57:21 +03:00
eladash 801e6114b6 rsx: Use relaxed store on fifo ctrl registers 2019-03-31 14:57:21 +03:00
eladash a3f65084df Fix sys_process_exit2 when SPUs are at av handler 2019-03-31 14:57:21 +03:00
eladash 1ed2055ec1 Fix cellVdecGetPicItem element popping behaviour 2019-03-31 14:57:21 +03:00
eladash 16f86b2458 Fix page fault area range check
Range end check was bugged (always true), also rewrite it so vm::get would be called only once
2019-03-31 14:57:21 +03:00
eladash f2bbae9db4 Remove handle in cellVdecClose 2019-03-31 14:57:21 +03:00
eladash 8eb59271a5 Improve error checking of cellVdecOpen
Those are the initial argument checks done by the firmware
2019-03-31 14:57:21 +03:00
eladash d6025c6764 Fix cellPadGetInfo port status returned
ASSIGN_CHANGES flag is not returned in this func.
2019-03-31 14:57:21 +03:00
eladash 47ca1b1dda Minor optimizations in cellPad
- Dont bother with shared_ptr since all pad_t management is going under the pad mutex.

- Change m_pads type into std::array since its size is known
2019-03-31 14:57:21 +03:00
eladash 6502d933df Fix stack memory view on the debugger
the debugger uses super ptr which was unmapped for stack.
2019-03-31 14:57:21 +03:00
scribam f15eb88f59 hle: Fix cellSysutilAvcExt module
And add cellSysutilAvcExtSetChatMode and cellSysutilAvcExtSetChatGroup
functions
2019-03-31 00:55:55 +03:00
scribam 1916cc1691 hle: Add cellSysmoduleUnloadModuleInternal and cellSysmoduleLoadModuleInternal functions 2019-03-30 23:52:56 +03:00
scribam f369aeab7a hle: Add FT_Get_X11_Font_Format function 2019-03-30 23:52:56 +03:00
scribam 0e9313d2df hle: Add cellFontInitLibraryFreeType function 2019-03-30 23:52:56 +03:00
Nekotekina d873802b9c Use LLVM 9
Use new add/sub with saturation intrinsics
2019-03-30 01:36:48 +03:00
Nekotekina 7e0b941e9f PPU LLVM: implement get_vrs<>() adaptor
Make use of structured bindings
2019-03-30 01:36:48 +03:00
Nekotekina d77fed6105 SPU LLVM: remove wrong dead code 2019-03-29 17:00:53 +03:00
scribam a254a203bb hle: Add libad_async and libad_core modules 2019-03-27 21:41:44 +00:00
Nekotekina 71b88cdc82 New SPU interpreter (SPU fast)
Use LLVM to build SPU interpreter.
Simplify interpreter loop.
2019-03-27 20:33:44 +03:00
scribam a9eb321814 hle: Add sceNpEulaAbort function 2019-03-26 23:19:01 +03:00
scribam 956d039415 hle: Add cellVideoPlayerUtility module 2019-03-24 19:16:49 +03:00
scribam 581b205f73 hle: Add cellPesmUtility module 2019-03-24 19:16:49 +03:00
scribam 6c40b9f3e0 hle: Add cellDtcpIpUtility module 2019-03-24 19:16:49 +03:00
scribam d6bf18eabc hle: Add some sceNpMatchingInt functions 2019-03-24 17:29:18 +03:00
scribam 32ae7e466c hle: Add cellNetAoi module 2019-03-24 17:29:18 +03:00
Nekotekina 7ea04d5d76 Minor optimization in SPU analyser
Reduce vector copy/allocation
2019-03-23 02:43:41 +03:00
Nekotekina 4b381fbbb1 Implement spu_runtime::reset
To handle JIT: Out Of Memory error.
2019-03-23 02:43:41 +03:00
Nekotekina 1880a17f79 SPU recs: implement spu_runtime::find
Use this function to link to existing functions from branch patchpoints.
Don't compile from branch patchpoints.
2019-03-23 02:43:41 +03:00
Nekotekina 31304f4234 SPU rec: refactor some trampoline generation
Move branch/dispatch trampoline generation at startup.
2019-03-23 02:43:41 +03:00
Nekotekina 3794f65bb6 Add cpu_flag::jit_return 2019-03-23 02:43:41 +03:00
Nekotekina 849411693a PPU LLVM: add MemoryManager3
For temporary allocations.
Add flags in jit_compiler constructor.
2019-03-23 02:43:41 +03:00
Nekotekina 466d58ccef SPU LLVM: fix branch patchpoints
Forgot to passthrough 3rd arg (rip)
2019-03-23 02:43:41 +03:00
kd-11 41b87cf577 rsx: Blit engine fixes
- If a transfer writes to a RTT and depth mismatch happens, create a local target and the upload function will likely resolve between the two
- If a surface is rejected, reset the target region!
2019-03-22 21:27:15 +03:00
kd-11 86ad204636 rsx: Rebase output region when using upload-fallback path 2019-03-22 21:27:15 +03:00
kd-11 dbc8e70ddd rsx: Silence some compiler noise 2019-03-22 21:27:15 +03:00
kd-11 3a4e3fa53a rsx: Fix use-after-modify condition when inserting a draw command out of order
- Fixes barrier->range rebase after the insert
2019-03-22 21:27:15 +03:00
kd-11 d731c07ade vk: Fix typeless resource management
- Fixes bugs that appear with high resolution scaling
2019-03-22 21:27:15 +03:00
kd-11 adc59f9810 rsx: Fix blit transfers when texel sizes mismatch
- Also refactors some bpp handling code
- Simplify texture intersection test to use a normalized/uniform coordinate space
- Fix broken bounds checking as well
2019-03-22 21:27:15 +03:00
kd-11 b879b32271 rsx: Fix bpp calculation taking resolution scaling into account
- Do not rely on image->width(), use surface_width() instead for unscaled values
- Refactor/clean GL rendertarget class a bit
2019-03-20 10:05:54 +03:00
kd-11 03fca73cf4 rsx: Fix blit intersection falling outside the available texture
- Just becaue we have a hit inside the tile of interest does not guarantee that it sits inside the texture!
2019-03-20 10:05:54 +03:00
Jan Beich 1a13d3ae4e OpenBSD doesn't have mcontext_t
`<signal.h>` provides `typedef struct sigcontext ucontext_t`
2019-03-19 22:24:49 +03:00
Jan Beich 1f845a3634 Don't try to build libusb on FreeBSD
CMake Error at 3rdparty/libusb/libusb/os/CMakeLists.txt:90 (message):
  Unsupported platform FreeBSD.  Currently only support Windows, OSX, &
  Linux.
2019-03-19 22:24:49 +03:00
Matias Morán 3eb2bcad39 Update DualShock 3 warning 2019-03-18 22:46:02 +00:00
RipleyTom 63bbe459ea DS3 pad handler 2019-03-18 19:05:02 +03:00
RipleyTom f2aaad11bb Adds libusb 2019-03-18 19:05:02 +03:00
kd-11 3ef16bee47 rsx: Fix texture lookups and avoid out-of-bounds copies/transfers 2019-03-17 21:50:11 +03:00
kd-11 bb65e45614 rsx: Implement GPU acceleration for rotated images 2019-03-17 21:50:11 +03:00
kd-11 5260f4b47d rsx: Improvements to memory flush mechanism
- Batch dma transfers whenever possible and do them in one go
- vk: Always ensure that queued dma transfers are visible to the GPU before they are needed by the host
  Requires a little refactoring to allow proper communication of the commandbuffer state
- vk: Code cleanup, the simplified mechanism makes it so that its not necessary to pass tons of args to methods
- vk: Fixup - do not forcefully do dma transfers on sections in an invalidation zone! They may have been speculated correctly already
2019-03-17 21:50:11 +03:00
kd-11 385485204b vk/gl: Omit unlocked data when grabbing flip sources from texture cache 2019-03-17 21:50:11 +03:00
kd-11 74eeacd091 vk/gl: Improve memory tag sync and test
- Properly pass parameters such as rsx-pitch to the surface store
- Do not crash if a surface fails verification in flip, use fall-back instead
2019-03-17 21:50:11 +03:00
kd-11 1a44446250 rsx: Fix dst upload block region
- The section needed starts at image origin, not transfer origin!
2019-03-17 21:50:11 +03:00
kd-11 a49a0f2a86 vk/gl: Synchronization improvements
- Properly wait for the buffer transfer operation to finish before map/readback!
- Change vkFence to vkEvent which works more like a GL fence which is what is needed.
- Implement supporting methods and functions
- Do not destroy fence by immediately waiting after copying to dma buffer
2019-03-17 21:50:11 +03:00
kd-11 85cb703633 rsx/cache: Debugging bugs introduced by the atlas coverage check
- Figured out why it breaks things, ofc can't actually check for coverage when there is no proper fbo data persistence
2019-03-17 21:50:11 +03:00
kd-11 3a4083263e rsx: Fix texture transfer when pitch does not match exactly 2019-03-17 21:50:11 +03:00
kd-11 612160a8ff rsx: Fix zero-pitch textures
- Assumption here is that only texel (0, 0) is accessible. Inline with other pitch 0 operations.
- TODO: Verify pitch 0 does not advance in Y either
2019-03-17 21:50:11 +03:00
kd-11 17c49d21a5 rsx/blit: Remove workarounds/hacks added for master. Start implementation/stubs for blit engine rotations in GPU 2019-03-17 21:50:11 +03:00
kd-11 745f8f9627 rsx: Remove pointless assert 2019-03-17 21:50:11 +03:00
Nekotekina e9b6beadfc SPU LLVM: implement static branch weights
May help branch prediction in some cases
2019-03-13 21:14:55 +03:00
Nekotekina 388d49db80 SPU LLVM: fix SPU MMIO in TSX mode 2019-03-13 21:14:55 +03:00
RipleyTom 980c3d6fe8 Fix lf_queue destructor 2019-03-13 15:59:13 +03:00
Ani 3b01e59cd4 InstallPup: Fix exception on newline trim
Avoid throwing fatal error when installing firmware without a \n character on the version string
Fixes #5646
2019-03-13 03:45:06 +03:00
Nekotekina a08cd0fe38 Update asmjit 2019-03-13 01:56:43 +03:00
Nekotekina 8623899499 Update 3rdparty/libpng 2019-03-13 01:34:55 +03:00
Nekotekina 6d4c9f2d41 Update 3rdparty/yaml-cpp 2019-03-13 01:33:23 +03:00
Nekotekina 30636f0199 Update 3rdparty/xxHash 2019-03-13 01:32:00 +03:00
Nekotekina 0aa9243557 Update 3rdparty/pugixml 2019-03-13 01:29:31 +03:00
Nekotekina eae5b9e049 Rewrite mulh64/umulh64
Use __int128_t/__uint128_t idioms on GCC/Clang.
May result in better codegen by avoiding inline asm.
2019-03-13 00:16:43 +03:00
Nekotekina 688aabc6c9 Add _sys_lwmutex_unlock2 syscall name 2019-03-12 23:55:13 +03:00
eladash 4a28319edf Implement SPU page faults notifications
* Implement both RawSPU and threaded SPU page fault recovery
* Guard page_fault_notification_entries access with a mutex
* Add missing lock in sys_ppu_thread_recover_page_fault/get_page_fault_context
* Fix EINVAL check in sys_ppu_thread_recover_page_fault, previously when the event was not found begin() was erased and
CELL_OK was returned.
* Fixed page fault recovery waiting logic:
- Do not rely on a single thread_ctrl notification (unsafe)
- Avoided a race where ::awake(ppu) can be called before ::sleep(ppu) therefore nop-ing out the notification
* Avoid inconsistencies with vm flags on page fault cause detection
* Fix sys_mmapper_enable_page_fault_notification EBUSY check
from RE it's allowed to register the same queue twice (on a different area) but not to enable page fault notifications twice
2019-03-12 13:28:31 +03:00
kd-11 1875dc3f18 gl: Fix buffer size calculations 2019-03-10 16:09:05 +03:00
kd-11 358558aaa7 cleanup and fixups 2019-03-10 16:09:05 +03:00
kd-11 04dda44225 rsx: Properly generate render target data with all parameters provided
- Build-up to variable-sized framebuffers and AA implementation
- Also allows accurate range calculation for our hit testing
2019-03-10 16:09:05 +03:00
kd-11 21bc6c7a87 rsx: Properly resolve data for upload when needed.
- Avoids blindly reusing blit dst sections as they may contain garbage.
  If a section was unlocked for a flush, just discard it as its reuse introduces potential data corruption.
  Since the data needs to be reuploaded anyway (for now), its better to start afresh
- In case of format mismatch, reset the calculated dst block
- Add a bounds check to determine if data contained in an atlas is good enough for sampling the cache.
  If not enough data is provided, fall back to full upload
2019-03-10 16:09:05 +03:00
kd-11 9d4d3d9443 rsx: Reimplement render target intersection tests when using hw accelerated blit engine
- Properly collapse memory tree when scanning in case of overlaps!
2019-03-10 16:09:05 +03:00
kd-11 f4ebcb0029 rsx: Properly decode packed renders from the type flag
- Seems to occupy bits [8-9]
2019-03-10 16:09:05 +03:00
kd-11 7c379432dd rsx: Implement proper pitch compatibility lookup
- When a single row is required or is all that is available, pitch has
no meaning as the coordinate space changed to 1D
2019-03-10 16:09:05 +03:00
kd-11 dccb4a4888 rsx/texture_cache: fixes to commit_framebuffer_memory 2019-03-10 16:09:05 +03:00
kd-11 b9e7b085fe rsx/texture_cache: Fixups for local resource hit and fast-path added 2019-03-10 16:09:05 +03:00
kd-11 a80f1a6ed4 gl: Fix memory tag sampling
- Also fixes a bad arg passed to glClearBuffer
2019-03-10 16:09:05 +03:00
kd-11 0395fb9955 rsx/tecture_cache: Addendum - fix data cast with scaling conversion (AA emulation)
- Blit operations do format conversion automatically which is NOT what we want!
- Scale onto temp buffer with similar format before performing data cast.
2019-03-10 16:09:05 +03:00
kd-11 10dc3dadee rsx/texture_cache: Improve framebuffer memory locking when WCB/WDB is not enabled
- Adds a new mode that removes non-framebuffer stuff inside framebuffer range
2019-03-10 16:09:05 +03:00
kd-11 563e205a72 rsx/texture_cache: Fix 'AA' scaling hack and restore collection template selection 2019-03-10 16:09:05 +03:00
kd-11 fa628f0ac4 rsx/surface_store: More aggressive tag sampling
- Use a 5-point tap with an X pattern across the target's memory space to reduce chances of false positives
- TODO: Potential false positives identified, requires some minor
restructuring of surface_store
2019-03-10 16:09:05 +03:00
kd-11 3a071a9c07 rsx: Texture search rewrite
- Perform a full search across all resource types as needed without
taking too many shortcuts/hacks
2019-03-10 16:09:05 +03:00
kd-11 6ef9dcd62e rsx: Handle mismatched/invalidated framebuffer sections when WCB is enabled 2019-03-10 16:09:05 +03:00
kd-11 ef071ebb6b rsx: Synchronize surface cache and texture cache data
- TODO: The whole upload_texture thing is a big hack, fix it properly
2019-03-10 16:09:05 +03:00
eladash a43e7c172c Fix shared memory page flags
TODO: From hw testing, it seems like sys_memory_get_page_attribute and sys_rsx_context_iomap check page size a little differently

get_page_attribute() always go by area flags, sys_rsx_context_iomap checks page by the page granularity
This means that if the area page size 64k, but shared memory is mapped with SYS_MEMORY_GRANULARITY_1M
It can be mapped for rsxio, but the page attribute will indicate 64k page size :thonk:
rsxio memory is verified to need 1m pages.
2019-03-08 23:44:46 +03:00
eladash 7470388e5a Use error_code in sys_rsx 2019-03-08 23:44:46 +03:00
eladash 26bcd0a4de Small improvements to sys_event_flag
- From RE, only protocols SYS_SYNC_FIFO and SYS_SYNC_PRIORITY are valid
- Use conditional atomic op store in a few places
- Properly revert changes in sys_event_flag_set when aomic op fails
2019-03-08 23:44:46 +03:00
elad bd259c8ae4 vulkan zcull: Fix deadlock in zcull flush waiting
Block adding additional flush requests until the first ones are treated (by adding missing lock)
2019-03-08 23:44:46 +03:00
elad fc253165e2 Correctness fix for RSXIOMem
- Make RSXIOMem volatile.
- Hint the compiler to check only once the address returned.
2019-03-08 23:44:46 +03:00
elad b7da3ea5cd Release ppu thread before ShowSaveDataDialog, Fixes #4031 2019-03-08 23:44:46 +03:00
elad ce8c92262d Treat X8R8G8B8 format as A8R8G8B8 in image_in, Fixes #5510 2019-03-08 23:44:46 +03:00
elad f272a5f779 sys_lwmutex fixup after #5680
sys_lwcond_wait unlocks always with the 'usual' unlocking flags
2019-03-08 23:44:46 +03:00
RipleyTom 61bd2ea799 Adds VID/PID for Guitar Hero guitar & drum 2019-03-08 17:52:48 +00:00
elad 3c9f03968c Yield before flushing io buffers in fsync (sys_fs) (#5506) 2019-03-08 16:07:14 +00:00
Nekotekina 4ea76def7c Update sys_lwmutex_lock and sys_lwmutex_unlock (liblv2 HLE)
Implement missing SYS_SYNC_RETRY logic
Following #5680
2019-03-06 15:09:50 +03:00
Nekotekina 986c750fdc VFS: fix sys_fs_opendir on root 2019-03-05 22:12:01 +03:00
Nekotekina efe067bfa0 LLVM: handle cascadelake CPU name 2019-03-05 21:47:34 +03:00
eladash e38b7aee5a check address in sys_rsx_context_iomap
* Fix 0 vm page flags to behave like 1m flags, follows c8a681e60
* check if address exists and valid for rsx io allcations (must be allocated on 1m pages)
2019-03-05 21:23:24 +03:00
eladash d82362fa1d Use sys_memory_allocate on rsx replayer to fix it 2019-03-05 21:23:24 +03:00
Nekotekina ebb24375d1 LLVM: handle icelake-client and icelake-server CPU names
Correct AVX / AVX-512 disabling logic
2019-03-01 00:20:04 +03:00
Nekotekina fb64b28886 SPU LLVM: reintroduce branch patchpoints
Previously only used on SPU ASMJIT, may improve perf in some cases.
Now refactored to spu_runtime::make_branch_patchpoint.
2019-03-01 00:08:20 +03:00
Nekotekina 7f6a410770 Add dummy __has_builtin macro, use rotate builtins if possible 2019-03-01 00:08:19 +03:00
Nekotekina 765d15f23f Optimize SPU trampolines
Load values in EAX and reuse it if possible
2019-03-01 00:08:19 +03:00
Nekotekina f143035af1 Fix sys_spu_thread_group_join wait condition
After waiting, thread group cannot be safely accessed
Following #5643
2019-03-01 00:08:19 +03:00
Nekotekina 9ee2867247 Improve count leading/trailing zeros implementation
Use x86 intrinsics if compiled with appropriate instruction support
2019-03-01 00:08:19 +03:00
Nekotekina 7b344b7654 Implement utils::refptr for typemap (with && syntax)
Ref-counted pointer, object is always allocated on heap.
Rvalue reference && in typemap is converted to refptr for convenience.
2019-03-01 00:08:19 +03:00
Nekotekina 6c5d9fffaa typemap: improve pointer support
If stored object is a pointer, access dereferenced object directly via ->
2019-03-01 00:08:19 +03:00
Jan Beich 240ba39d16 Chase GIT_VERSION rename after 923700a167 2019-02-28 04:48:16 +03:00
RipleyTom de5379a69f Static hle implementation 2019-02-27 22:54:59 +03:00
eladash a22297f205 exception throwing fix in sys_lwmutex_create
arg6 doesnt exist, if arg4 is not negative name is discarded and treated as 0.
2019-02-27 22:16:08 +03:00
eladash d4459af4b3 Implement _sys_lwmutex_unlock (SYS_SYNC_RETRY mode) 2019-02-27 22:16:08 +03:00
RipleyTom ad6b0ee122 Adds class type to controller options 2019-02-27 18:13:19 +00:00
Megamouse b107718869 sysPrxForUser: improve crash dump functions
this might fix some crashes that could appear in the todo logging itself
2019-02-26 21:53:59 +00:00
Alex James f5163cc4ff CMake: Use Info.plist template for macOS bundle
Define and use a template for Info.plist for the macOS bundle. The template
uses the previously defined RPCS3_GIT_TAG variable for automatically
generating the version strings, and also enables proper HiDPI support.
2019-02-26 21:09:18 +00:00
Alex James 923700a167 CMake: Refactor git-version.cmake
Refactor git-version.cmake to define the RPCS3_GIT_VERSION,
RPCS3_GIT_BRANCH, and RPCS3_GIT_TAG variables, and move the generation
of git-version.h to a function (gen_git_version).
2019-02-26 21:09:18 +00:00
JohnHolmesII c325534f07 readme - direct qt link to avoid confused kot noises 2019-02-23 13:29:38 +01:00
Megamouse 14ddb0c5b4 Qt: hide viewport settings on custom configs 2019-02-21 22:02:02 +01:00
German 4c72f7c1de Fix clear string container in CgBinaryFragmentProgram.cpp 2019-02-18 16:34:16 +03:00
German 3b9f9dd4c5 Fix true clear string container in GameInfo.h 2019-02-18 16:34:16 +03:00
German 45c31a99a3 Fix true clear string container in PPUModule.cpp 2019-02-18 16:34:16 +03:00
elad 63a9421634 Fix race in sys_lwcond_wait on error code 2019-02-16 21:41:59 +03:00
EmulationChannel 92ff7c4c24 Update FW Latest Version
Updates the latest FW 4.84 version according to: https://www.playstation.com/en-us/support/system-updates/ps3/
2019-02-16 13:59:30 +01:00
Filippe LeMarchand 56e24f8993 Update rpcs3.appdata.xml
- split description to paragraphs
- add bugtracker and quickstart urls
- add screenshot
2019-02-14 03:10:36 +03:00
Filippe LeMarchand a8621ebc22 Remove zero-width spaces (u200b) 2019-02-14 03:10:36 +03:00
Megamouse d4888a4973 cellSysCacheMount: don't return RET_OK_RELAYED on empty cacheId
The system cache is supposed to be cleared but I don't think we wanna do that
2019-02-13 01:55:07 +03:00
Megamouse 4a1499e0be cellMsgDialog: optionally make dialogs blocking and fix exit condition
and apply review fixes
2019-02-12 21:06:10 +03:00
Megamouse fe79e541dd cellGame: improve exit functions 2019-02-12 21:06:10 +03:00
Megamouse 17a5e0bc98 cellGame: add error_code 2019-02-12 21:06:10 +03:00
Megamouse 982b2ab88f Qt: only show supported resolutions in custom configs 2019-02-11 20:34:33 +01:00
eladash d6995f40c7 Fixup for sys_lwcond_signal_x error checking 2019-02-11 01:13:29 +03:00
eladash fa647bc121 Fix race condion in sys_spu_thread_group_join 2019-02-10 18:20:24 +03:00
eladash 84d42ecb65 Add EFAULT checks to spu_thread_group_join, ppu_thread_join
Order of checks is based on firmware
2019-02-10 00:16:57 +03:00
eladash 0861226271 Make more use of the new atomic_t<>::release 2019-02-10 00:16:57 +03:00
eladash e3ee481f01 Make sys_spu_thread_group_join return once per termination 2019-02-10 00:16:57 +03:00
kd-11 19ff95da70 vk: Fix usage of VK_IMAGE_LAYOUT_GENERAL
- Properly synchronize when transitioning to/from GENERAL layout.
- General layout requires full pipeline dependency since its used in a 'general' sense. As such, its use is to be largely avoided.
2019-02-07 11:40:17 +03:00
kd-11 38887bc03e gl/vk: Improvements to overlay rendering
- gl: Properly initialize and manage sampler states
- gl/vk: Snap overlay elements to pixel grid by aligning to pixel centers
- overlays: Disable grid snapping in stb since its now handled in the backend
2019-02-05 12:15:12 +03:00
kd-11 4c593959fd overlays/save_dialog: Layout improvements
- Make detail a separate text entity as it often contains a lot of noise
- Properly pad the entry if needed to avoid text sitting too close to the edge
2019-02-03 22:26:46 +03:00
kd-11 67cdec577f overlays/util: Add support for glyph set lowering when mapping utf8 to ascii8
- Lower fullwidth glyphs to halfwidth counterparts
- Lower CJK punctuation glyphs
- Lower general punctuation glyphs
2019-02-03 22:26:46 +03:00
kd-11 a36d3af3b4 vk: Minor frame management improvements 2019-02-02 11:54:01 +03:00
kd-11 27af05da1a osk: Fixup attempt for hang in close callback where a sysutil_callback fails to fire. 2019-02-02 11:54:01 +03:00
kd-11 b36cb66129 overlays: Allow use of extended ascii8
- Use custom string conversion to ensure overlay deals with extended ascii whenever possible
- Improves language compatibility greatly and avoids empty spaces for unknown glyphs
2019-02-02 11:54:01 +03:00
kd-11 12990f3ca3 overlays/util: Strip extended codes from utf-16 encoded strings 2019-02-02 11:54:01 +03:00
kd-11 9e39e2d2c4 gl/vk: Fix clip region scaling for overlay elements 2019-02-02 11:54:01 +03:00
kd-11 3653c2eb0d overlays/osk: Add support for edit text control and disabled cells
- Allows to disable cells from being selectable.
- Edit text control adds proper support for multiline and a functioning caret
2019-02-02 11:54:01 +03:00
kd-11 faf5221b0d overlays: Implement edit_text control 2019-02-02 11:54:01 +03:00
kd-11 c434e0ce27 overlays/osk: Add more buttons to native dialog and other improvements
- Adds all the major buttons to native dialog input options
- Adds more button options for the native osk
- Brighten osk cell backgrounds a bit to improve visibility
2019-02-02 11:54:01 +03:00
kd-11 9ed9d7e947 overlays/osk: Implement native osk interface 2019-02-02 11:54:01 +03:00
kd-11 9d4b19b97a vk: Increase number of draw calls per frame for overlays to 1024
- Allows for more complex interface design
2019-02-02 11:54:01 +03:00
Megamouse 8920eda321 facepalm: fix open_config_dir old vs. new location switch up 2019-02-01 17:47:27 +03:00
Ani f4afd56e67 rpcs3_version: Bump to 0.0.6 2019-01-31 22:08:45 +00:00
kd-11 f47d3a761b vk: Hotfix for fullscreen not working on non-windows platforms 2019-02-01 00:22:11 +03:00
1054 changed files with 172153 additions and 154916 deletions
+66
View File
@@ -0,0 +1,66 @@
#!/bin/sh -ex
# Setup Qt variables
export QT_BASE_DIR=/opt/qt${QTVERMIN}
export PATH=$QT_BASE_DIR/bin:$PATH
export LD_LIBRARY_PATH=$QT_BASE_DIR/lib/x86_64-linux-gnu:$QT_BASE_DIR/lib
cd rpcs3 || exit 1
# Pull all the submodules except llvm, since it is built separately and we just download that build
# Note: Tried to use git submodule status, but it takes over 20 seconds
# shellcheck disable=SC2046
git submodule -q update --init $(awk '/path/ && !/llvm/ { print $3 }' .gitmodules)
# Download pre-compiled llvm libs
curl -sLO https://github.com/RPCS3/llvm-mirror/releases/download/custom-build/llvmlibs-linux.tar.gz
mkdir llvmlibs
tar -xzf ./llvmlibs-linux.tar.gz -C llvmlibs
mkdir build && cd build || exit 1
if [ "$COMPILER" = "gcc" ]; then
# These are set in the dockerfile
export CC=${GCC_BINARY}
export CXX=${GXX_BINARY}
export LINKER=gold
# We need to set the following variables for LTO to link properly
export AR=/usr/bin/gcc-ar-9
export RANLIB=/usr/bin/gcc-ranlib-9
export CFLAGS="-fuse-linker-plugin"
else
export CC=${CLANG_BINARY}
export CXX=${CLANGXX_BINARY}
export LINKER=lld
export AR=/usr/bin/llvm-ar-$LLVMVER
export RANLIB=/usr/bin/llvm-ranlib-$LLVMVER
fi
export CFLAGS="$CFLAGS -fuse-ld=${LINKER}"
cmake .. \
-DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_LLVM_SUBMODULE=OFF \
-DLLVM_DIR=llvmlibs/lib/cmake/llvm/ \
-DUSE_NATIVE_INSTRUCTIONS=OFF \
-DUSE_PRECOMPILED_HEADERS=OFF \
-DCMAKE_C_FLAGS="$CFLAGS" \
-DCMAKE_CXX_FLAGS="$CFLAGS" \
-DCMAKE_AR="$AR" \
-DCMAKE_RANLIB="$RANLIB" \
-G Ninja
ninja; build_status=$?;
cd ..
# If it compiled succesfully let's deploy depending on the build pipeline (Travis, Azure Pipelines).
# Travis only deploys on master, and it publishes to GitHub releases. Azure publishes PRs as artifacts
# only.
{ [ "$IS_AZURE" = "true" ] ||
{ [ "$TRAVIS_BRANCH" = "master" ] && [ "$TRAVIS_PULL_REQUEST" = "false" ]; };
} && SHOULD_DEPLOY="true" || SHOULD_DEPLOY="false"
if [ "$build_status" -eq 0 ] && [ "$SHOULD_DEPLOY" = "true" ]; then
.ci/deploy-linux.sh
fi
+11 -10
View File
@@ -1,26 +1,27 @@
#!/bin/sh -ex
export CCACHE_SLOPPINESS=pch_defines,time_macros
export CMAKE_PREFIX_PATH=/usr/local/opt/qt5/
export PATH="/usr/local/opt/ccache/libexec:$PATH"
# Allow using optional on 10.13
sudo perl -pi -e 'BEGIN{undef $/;} s/(_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS[ ]+\\\n[^\n]+10)\.14/\1.13/;' \
$(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__config
# Setup vulkan and gfx-rs/portability
curl -sLO https://github.com/gfx-rs/portability/releases/download/latest/gfx-portability-macos-latest.zip
unzip -: gfx-portability-macos-latest.zip
curl -sLO https://github.com/KhronosGroup/Vulkan-Headers/archive/sdk-1.1.92.0.zip
curl -sLO https://github.com/KhronosGroup/Vulkan-Headers/archive/sdk-1.1.106.0.zip
unzip -: sdk-*.zip
mkdir vulkan-sdk
ln -s ${PWD}/Vulkan-Headers*/include vulkan-sdk/include
ln -s "${PWD}"/Vulkan-Headers*/include vulkan-sdk/include
mkdir vulkan-sdk/lib
cp target/release/libportability.dylib vulkan-sdk/lib/libVulkan.dylib
# Let macdeployqt locate and install Vulkan library
install_name_tool -id ${PWD}/vulkan-sdk/lib/libVulkan.dylib vulkan-sdk/lib/libVulkan.dylib
export VULKAN_SDK=${PWD}/vulkan-sdk
install_name_tool -id "${PWD}"/vulkan-sdk/lib/libVulkan.dylib vulkan-sdk/lib/libVulkan.dylib
export VULKAN_SDK="${PWD}/vulkan-sdk"
git submodule update --quiet --init asmjit 3rdparty/ffmpeg 3rdparty/pugixml 3rdparty/GSL 3rdparty/libpng 3rdparty/cereal 3rdparty/hidapi 3rdparty/xxHash 3rdparty/yaml-cpp Vulkan/glslang
# Pull all the submodules except llvm
# Note: Tried to use git submodule status, but it takes over 20 seconds
# shellcheck disable=SC2046
git submodule -q update --init $(awk '/path/ && !/llvm/ { print $3 }' .gitmodules)
mkdir build; cd build
mkdir build && cd build || exit 1
cmake .. -DWITH_LLVM=OFF -DUSE_NATIVE_INSTRUCTIONS=OFF -G Ninja
ninja
+67
View File
@@ -0,0 +1,67 @@
#!/bin/sh -ex
cd build || exit 1
if [ "$DEPLOY_APPIMAGE" = "true" ]; then
DESTDIR=appdir ninja install
QT_APPIMAGE="linuxdeployqt.AppImage"
curl -sL "https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage" > "$QT_APPIMAGE"
chmod a+x "$QT_APPIMAGE"
"./$QT_APPIMAGE" --appimage-extract
./squashfs-root/AppRun ./appdir/usr/share/applications/*.desktop -bundle-non-qt-libs
ls ./appdir/usr/lib/
rm -r ./appdir/usr/share/doc
rm ./appdir/usr/lib/libxcb*
cp "$(readlink -f /lib/x86_64-linux-gnu/libnsl.so.1)" ./appdir/usr/lib/libnsl.so.1
export PATH=/rpcs3/build/squashfs-root/usr/bin/:${PATH}
# Embed newer libstdc++ for distros that don't come with it (ubuntu 16.04)
mkdir -p appdir/usr/optional/ ; mkdir -p appdir/usr/optional/libstdc++/
cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ./appdir/usr/optional/libstdc++/
rm ./appdir/AppRun
curl -sL https://github.com/RPCS3/AppImageKit-checkrt/releases/download/continuous2/AppRun-patched-x86_64 -o ./appdir/AppRun
chmod a+x ./appdir/AppRun
curl -sL https://github.com/RPCS3/AppImageKit-checkrt/releases/download/continuous2/exec-x86_64.so -o ./appdir/usr/optional/exec.so
# Compile checker binary for AppImageKit-checkrt
# This may need updating if you update the compiler or rpcs3 uses newer c++ features
# See https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/config/abi/pre/gnu.ver
# for which definitions correlate to which CXXABI version.
# Currently we target a minimum of GLIBCXX_3.4.26 and CXXABI_1.3.11
printf "#include <bits/stdc++.h>\nint main(){std::make_exception_ptr(0);std::pmr::get_default_resource();}" | $CXX -x c++ -std=c++2a -o ./appdir/usr/optional/checker -
# Package it up and send it off
./squashfs-root/usr/bin/appimagetool /rpcs3/build/appdir
ls
COMM_TAG="$(grep 'version{.*}' ../rpcs3/rpcs3_version.cpp | awk -F[\{,] '{printf "%d.%d.%d", $2, $3, $4}')"
COMM_COUNT="$(git rev-list --count HEAD)"
COMM_HASH="$(git rev-parse --short=8 HEAD)"
RPCS3_APPIMAGE="rpcs3-v${COMM_TAG}-${COMM_COUNT}-${COMM_HASH}_linux64.AppImage"
curl -sLO https://github.com/hcorion/uploadtool/raw/master/upload.sh
mv ./RPCS3*.AppImage "$RPCS3_APPIMAGE"
# If we're building using Azure Pipelines, let's copy over the AppImage artifact
if [ -n "$BUILD_ARTIFACTSTAGINGDIRECTORY" ]; then
cp "$RPCS3_APPIMAGE" ~/artifacts
fi
FILESIZE=$(stat -c %s ./rpcs3*.AppImage)
SHA256SUM=$(sha256sum ./rpcs3*.AppImage)
if [ -n "$GITHUB_TOKEN" ]; then
unset TRAVIS_REPO_SLUG
REPO_SLUG=RPCS3/rpcs3-binaries-linux \
UPLOADTOOL_BODY="$SHA256SUM;${FILESIZE}B"\
RELEASE_NAME=build-${TRAVIS_COMMIT}\
RELEASE_TITLE=${COMM_TAG}-${COMM_COUNT}\
REPO_COMMIT=d812f1254a1157c80fd402f94446310560f54e5f\
bash upload.sh rpcs3*.AppImage
fi
fi
if [ "$DEPLOY_PPA" = "true" ]; then
export DEBFULLNAME="RPCS3 Build Bot"
fi
+24
View File
@@ -0,0 +1,24 @@
#!/bin/sh -ex
# BUILD_blablabla is Azure specific, so we wrap it for portability
ARTIFACT_DIR="$BUILD_ARTIFACTSTAGINGDIRECTORY"
# Remove unecessary files
rm -f ./bin/rpcs3.exp ./bin/rpcs3.lib ./bin/rpcs3.pdb
# Prepare compatibility database for packaging, as well as
# certificate for ssl (auto-updater)
curl -sL 'https://rpcs3.net/compatibility?api=v1&export' | iconv -t UTF-8 > ./bin/GuiConfigs/compat_database.dat
curl -sL 'https://curl.haxx.se/ca/cacert.pem' > ./bin/cacert.pem
# Package artifacts
7z a -m0=LZMA2 -mx9 "$BUILD" ./bin/*
# Generate sha256 hashes
# Write to file for GitHub releases
sha256sum "$BUILD" | awk '{ print $1 }' | tee "$BUILD.sha256"
echo "$(cat "$BUILD.sha256");$(stat -c %s "$BUILD")B" > GitHubReleaseMessage.txt
# Move files to publishing directory
mv -- "$BUILD" "$ARTIFACT_DIR"
mv -- "$BUILD.sha256" "$ARTIFACT_DIR"
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh -e
# Export variables for later stages of the Azure pipeline
# Values done in this manner will appear as environment variables
# in later stages.
# From pure-sh-bible
# Setting 'IFS' tells 'read' where to split the string.
while IFS='=' read -r key val; do
# Skip over lines containing comments.
[ "${key##\#*}" ] || continue
echo "##vso[task.setvariable variable=$key]$val"
done < ".ci/azure-vars.env"
+114
View File
@@ -0,0 +1,114 @@
#!/bin/sh -ex
# These are Azure specific, so we wrap them for portability
REPO_NAME="$BUILD_REPOSITORY_NAME"
REPO_BRANCH="$SYSTEM_PULLREQUEST_SOURCEBRANCH"
PR_NUMBER="$SYSTEM_PULLREQUEST_PULLREQUESTID"
# Resource/dependency URLs
# Qt mirrors can be volatile and slow, so we list 2
#QT_HOST="http://mirrors.ocf.berkeley.edu/qt/"
QT_HOST="http://qt.mirror.constant.com/"
QT_URL_VER=$(echo "$QT_VER" | sed "s/\.//g")
QT_PREFIX="online/qtsdkrepository/windows_x86/desktop/qt5_${QT_URL_VER}/qt.qt5.${QT_URL_VER}.win64_msvc2017_64/${QT_VER}-0-${QT_DATE}"
QT_SUFFIX="-Windows-Windows_10-MSVC2017-Windows-Windows_10-X86_64.7z"
QT_BASE_URL="${QT_HOST}${QT_PREFIX}qtbase${QT_SUFFIX}"
QT_WINE_URL="${QT_HOST}${QT_PREFIX}qtwinextras${QT_SUFFIX}"
QT_DECL_URL="${QT_HOST}${QT_PREFIX}qtdeclarative${QT_SUFFIX}"
QT_TOOL_URL="${QT_HOST}${QT_PREFIX}qttools${QT_SUFFIX}"
LLVMLIBS_URL='https://github.com/RPCS3/llvm-mirror/releases/download/custom-build-win/llvmlibs_mt.7z'
GLSLANG_URL='https://github.com/RPCS3/glslang/releases/download/custom-build-win/glslanglibs_mt.7z'
VULKAN_SDK_URL="https://sdk.lunarg.com/sdk/download/${VULKAN_VER}/windows/VulkanSDK-${VULKAN_VER}-Installer.exe"
DEP_URLS=" \
$QT_BASE_URL \
$QT_WINE_URL \
$QT_DECL_URL \
$QT_TOOL_URL \
$LLVMLIBS_URL \
$GLSLANG_URL \
$VULKAN_SDK_URL"
# Azure pipelines doesn't make a cache dir if it doesn't exist, so we do it manually
[ -d "$CACHE_DIR" ] || mkdir "$CACHE_DIR"
# Pull all the submodules except llvm, since it is built separately and we just download that build
# Note: Tried to use git submodule status, but it takes over 20 seconds
# shellcheck disable=SC2046
git submodule -q update --init $(awk '/path/ && !/llvm/ { print $3 }' .gitmodules)
# Git bash doesn't have rev, so here it is
rev()
{
echo "$1" | awk '{ for(i = length($0); i != 0; --i) { a = a substr($0, i, 1); } } END { print a }'
}
# Usage: download_and_verify url checksum algo file
# Check to see if a file is already cached, and the checksum matches. If not, download it.
# Tries up to 3 times
download_and_verify()
{
url="$1"
correctChecksum="$2"
algo="$3"
fileName="$4"
for _ in 1 2 3; do
[ -e "$CACHE_DIR/$fileName" ] || curl -L -o "$CACHE_DIR/$fileName" "$url"
fileChecksum=$("${algo}sum" "$CACHE_DIR/$fileName" | awk '{ print $1 }')
[ "$fileChecksum" = "$correctChecksum" ] && return 0
rm "$CACHE_DIR/$fileName"
done
return 1;
}
# Some dependencies install here
[ -d "./lib" ] || mkdir "./lib"
for url in $DEP_URLS; do
# Get the filename from the URL. Breaks if urls have js args, so don't do that pls
fileName="$(rev "$(rev "$url" | cut -d'/' -f1)")"
[ -z "$fileName" ] && echo "Unable to parse url: $url" && exit 1
# shellcheck disable=SC1003
case "$url" in
*qt*) checksum=$(curl -L "${url}.sha1"); algo="sha1"; outDir='C:\Qt\' ;;
*llvm*) checksum=$(curl -L "${url}.sha256"); algo="sha256"; outDir="." ;;
*glslang*) checksum=$(curl -L "${url}.sha256"); algo="sha256"; outDir="./lib/Release - LLVM-x64" ;;
*Vulkan*)
# Vulkan setup needs to be run in batch environment
# Need to subshell this or else it doesn't wait
download_and_verify "$url" "$VULKAN_SDK_SHA" "sha256" "$fileName"
cp "$CACHE_DIR/$fileName" .
_=$(echo "$fileName /S" | cmd)
continue
;;
*) echo "Unknown url resource: $url"; exit 1 ;;
esac
download_and_verify "$url" "$checksum" "$algo" "$fileName"
7z x -y "$CACHE_DIR/$fileName" -aos -o"$outDir"
done
# Gather explicit version number and number of commits
COMM_TAG=$(awk '/version{.*}/ { printf("%d.%d.%d", $5, $6, $7) }' ./rpcs3/rpcs3_version.cpp)
COMM_COUNT=$(git rev-list --count HEAD)
COMM_HASH=$(git rev-parse --short=8 HEAD)
# Format the above into filenames
if [ -n "$PR_NUMBER" ]; then
AVVER="${COMM_TAG}-${COMM_HASH}"
BUILD="rpcs3-v${AVVER}_win64.7z"
else
AVVER="${COMM_TAG}-${COMM_COUNT}"
BUILD="rpcs3-v${AVVER}-${COMM_HASH}_win64.7z"
fi
# BRANCH is used for experimental build warnings for pr builds, used in main_window.cpp.
# BUILD is the name of the release artifact
# AVVER is used for GitHub releases, it is the version number.
BRANCH="${REPO_NAME}/${REPO_BRANCH}"
echo "BRANCH=$BRANCH" > .ci/azure-vars.env
echo "BUILD=$BUILD" >> .ci/azure-vars.env
echo "AVVER=$AVVER" >> .ci/azure-vars.env
+7
View File
@@ -2,6 +2,13 @@
TRAVIS_PULL_REQUEST
TRAVIS_BRANCH
TRAVIS_COMMIT
# Variables set by Azure Pipelines
IS_AZURE
BUILD_REASON
BUILD_SOURCEVERSION
BUILD_ARTIFACTSTAGINGDIRECTORY
BUILD_REPOSITORY_NAME
BUILD_SOURCEBRANCHNAME
# Variables for Travis build matrix
COMPILER
DEPLOY_APPIMAGE
+2
View File
@@ -0,0 +1,2 @@
patreon: Nekotekina
custom: https://rpcs3.net/alipay
-24
View File
@@ -1,24 +0,0 @@
<!---
THIS IS NOT A SUPPORT FORUM, FOR SUPPORT GO TO:
https://forums.rpcs3.net
or our discord:
https://discord.me/RPCS3
PLEASE READ THE GUIDELINES BEFORE OPENING AN ISSUE:
https://github.com/RPCS3/rpcs3/blob/master/.github/CONTRIBUTING.md
====================================================
When submitting an issue, please check the following:
- You have read the above.
- You have provided the version (commit hash) of RPCS3 you are using.
- You have provided sufficient detail for the issue to be reproduced.
- You have provided system specs.
- Please also provide:
- For crashes, a backtrace.
- For graphical issues, comparison screenshots with real hardware.
Remember that the GitHub Issue Tracker is not the place to ask for support or to submit Game Compatibility reports. You must use our forums for that.
--->
@@ -0,0 +1,28 @@
---
name: Regression report
about: If something worked before, but now it doesn't
title: ''
labels: ''
assignees: ''
---
<!---
Please do not ask for help or report compatibility regressions here, use RPCS3 Discord server or forums instead.
Please read Contributing guide under Helpful resources on the right. →
--->
## Quick summary
Please describe what exactly has stopped working correctly
## Details
Please provide _exact_ build (or commit) information that introduced the regression you're reporting.
Please see How to find the build that caused a regression in wiki: https://wiki.rpcs3.net/index.php?title=Help:Using_different_versions_of_RPCS3#How_to_find_the_build_that_caused_a_regression.3F
You can find old builds here: https://rpcs3.net/compatibility?b
Please remember to attach TWO logs: one for the build that exhibits regression, and one for the previous build that works as expected.
If the regression caused a graphical issue, please provide RSX capture and a Renderdoc capture that demonstrate the issue.
To create an RSX capture, enable Debugger view (View → Show Debugger) and use the RSX Capture button. Captues will be stored in RPCS3 folder → captures.
To create Renderdoc capture, please refer to Renderdoc documentation https://renderdoc.org/docs/how/how_capture_frame.html
+26
View File
@@ -0,0 +1,26 @@
---
name: Bug report
about: If something doesn't work correctly in RPCS3
title: ''
labels: ''
assignees: ''
---
<!---
Please do not ask for help or general inquiry here, use RPCS3 Discord server instead.
Please read Contributing guide under Helpful resources on the right. →
--->
## Quick summary
Please summarize the issue here
## Details
Please provide as much detail as you can here, including
- expected (real hw) and observed (emulator) behavior
- full logs (not from UI)
- back traces (for hard crashes)
- screenshots, Renderdoc, and RSX captures (for visual issues)
- your system configuration (OS, CPU, GPU, driver versions, etc)
- anything else you deem to be important
+11
View File
@@ -0,0 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: Quickstart guide
url: https://rpcs3.net/quickstart
about: Everything you need to know to install and configure emulator, and add games
- name: Ask for help
url: https://discord.me/RPCS3
about: If you have some questions or need help, please use our Discord server instead of GitHub
- name: Report game compatibility
url: https://forums.rpcs3.net/thread-196671.html
about: Please use RPCS3 forums to submit or update game compatibility status
+18
View File
@@ -37,6 +37,7 @@
/rpcs3/Debug
/rpcs3/Release
/llvm_build
/3rdparty/llvm_build
/Vulkan/Vulkan-build
/Vulkan/glslang-build
@@ -71,10 +72,15 @@ Makefile
*CMakeFiles*
CMakeCache.txt
*cmake_install.cmake*
CPackConfig.cmake
CPackSourceConfig.cmake
compile_commands.json
# cotire
rpcs3/cotire/*
rpcs3/rpcs3_*_cotire.cmake
rpcs3/Emu/rpcs3_emu_CXX_Release_cotire.cmake
rpcs3/Emu/rpcs3_emu_CXX_cotire.cmake
# kdevelop
*.kdev4
@@ -97,3 +103,15 @@ CMakeLists.txt.user
# macOS
.DS_Store
# 7zlib
/3rdparty/7z/**/*.lib
# yaml-cpp
yaml-cpp.pc
# libusb
/3rdparty/libusb_cmake/config.h
/3rdparty/libusb_cmake/libusb-1.0.pc
# ssl certificate
cacert.pem
+25 -9
View File
@@ -1,6 +1,6 @@
[submodule "rpcs3-ffmpeg"]
path = 3rdparty/ffmpeg
url = https://github.com/hrydgard/ppsspp-ffmpeg
url = https://github.com/RPCS3/ffmpeg-core
ignore = dirty
[submodule "asmjit"]
path = asmjit
@@ -8,12 +8,7 @@
ignore = dirty
[submodule "llvm"]
path = llvm
url = https://github.com/RPCS3/llvm
branch = release_60
ignore = dirty
[submodule "GSL"]
path = 3rdparty/GSL
url = https://github.com/Microsoft/GSL.git
url = https://github.com/RPCS3/llvm-mirror
ignore = dirty
[submodule "Vulkan/glslang"]
path = Vulkan/glslang
@@ -21,7 +16,7 @@
ignore = dirty
[submodule "3rdparty/cereal"]
path = 3rdparty/cereal
url = https://github.com/USCiLab/cereal.git
url = https://github.com/RPCS3/cereal.git
ignore = dirty
[submodule "3rdparty/zlib"]
path = 3rdparty/zlib
@@ -42,9 +37,30 @@
ignore = dirty
[submodule "3rdparty/yaml-cpp"]
path = 3rdparty/yaml-cpp
url = https://github.com/jbeder/yaml-cpp.git
url = https://github.com/RPCS3/yaml-cpp.git
ignore = dirty
[submodule "3rdparty/libpng"]
path = 3rdparty/libpng
url = https://github.com/glennrp/libpng.git
ignore = dirty
[submodule "3rdparty/libusb"]
path = 3rdparty/libusb
url = https://github.com/libusb/libusb.git
ignore = dirty
[submodule "3rdparty/FAudio"]
path = 3rdparty/FAudio
url = https://github.com/FNA-XNA/FAudio.git
ignore = dirty
[submodule "3rdparty/span"]
path = 3rdparty/span
url = https://github.com/tcbrindle/span
ignore = dirty
[submodule "3rdparty/curl"]
path = 3rdparty/curl
url = https://github.com/RipleyTom/curl.git
ignore = dirty
[submodule "3rdparty/wolfssl"]
path = 3rdparty/wolfssl
url = https://github.com/RipleyTom/wolfssl.git
ignore = dirty
+23 -19
View File
@@ -1,37 +1,41 @@
language: cpp
matrix:
jobs:
include:
- os: linux
dist: xenial
env:
- NAME="Linux build"
- COMPILER="gcc"
dist: trusty
- DEPLOY_APPIMAGE="true"
services: docker
cache: ccache
install: "docker pull rpcs3/rpcs3-travis-trusty:1.0"
script: 'travis_wait docker run -v $(pwd):/rpcs3 -v "$HOME/.ccache":/root/.ccache --env-file .travis/travis.env rpcs3/rpcs3-travis-trusty:1.0 /bin/bash -ex /rpcs3/.travis/build-linux.bash'
compiler: gcc
install: "docker pull rpcs3/rpcs3-travis-xenial:1.2"
script: 'travis_wait docker run -v $(pwd):/rpcs3 -v "$HOME/.ccache":/root/.ccache --env-file .ci/travis.env rpcs3/rpcs3-travis-xenial:1.2 /rpcs3/.ci/build-linux.sh'
- os: linux
dist: xenial
env:
- NAME="Linux build"
- COMPILER="clang"
- DEPLOY_APPIMAGE="true"
dist: trusty
services: docker
cache: ccache
install: "docker pull rpcs3/rpcs3-travis-trusty:1.0"
script: 'travis_wait docker run -v $(pwd):/rpcs3 -v "$HOME/.ccache":/root/.ccache --env-file .travis/travis.env rpcs3/rpcs3-travis-trusty:1.0 /bin/bash -ex /rpcs3/.travis/build-linux.bash'
compiler: clang
install: "docker pull rpcs3/rpcs3-travis-xenial:1.2"
script: 'travis_wait docker run -v $(pwd):/rpcs3 -v "$HOME/.ccache":/root/.ccache --env-file .ci/travis.env rpcs3/rpcs3-travis-xenial:1.2 /rpcs3/.ci/build-linux.sh'
# - os: osx
# osx_image: xcode11.3
# addons:
# homebrew:
# packages:
# - ccache
# - glew
# - ninja
# - qt
# script: "/bin/bash -ex .ci/build-mac.sh"
# cache: ccache
allow_failures:
- os: osx
osx_image: xcode10.1
addons:
homebrew:
packages:
- ccache
- glew
- ninja
- qt
script: "/bin/bash -ex .travis/build-mac.bash"
cache: ccache
git:
depth: false # Unshallow clone to obtain proper GIT_VERSION
submodules: false
-35
View File
@@ -1,35 +0,0 @@
#!/bin/env bash -ex
shopt -s nocasematch
# Setup Qt variables
export QT_BASE_DIR=/opt/qt${QTVERMIN}
export PATH=$QT_BASE_DIR/bin:$PATH
export LD_LIBRARY_PATH=$QT_BASE_DIR/lib/x86_64-linux-gnu:$QT_BASE_DIR/lib
cd rpcs3
git submodule update --quiet --init asmjit 3rdparty/ffmpeg 3rdparty/pugixml 3rdparty/GSL 3rdparty/libpng 3rdparty/cereal 3rdparty/hidapi 3rdparty/xxHash 3rdparty/yaml-cpp Vulkan/glslang
# Download pre-compiled llvm libs
curl -sLO https://github.com/RPCS3/llvm/releases/download/continuous-linux-master/llvmlibs-linux.tar.gz
mkdir llvmlibs
tar -xzf ./llvmlibs-linux.tar.gz -C llvmlibs
mkdir build ; cd build
if [ $COMPILER = "gcc" ]; then
# These are set in the dockerfile
export CC=${GCC_BINARY}
export CXX=${GXX_BINARY}
else
export CC=${CLANG_BINARY}
export CXX=${CLANGXX_BINARY}
fi
cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_LLVM_SUBMODULE=OFF -DLLVM_DIR=llvmlibs/lib/cmake/llvm/ -DUSE_NATIVE_INSTRUCTIONS=OFF -G Ninja
ninja
cd ..
# If it compiled succesfully let's deploy
if [ $? -eq 0 ] && [ -n "$GITHUB_TOKEN" ] && [ "$TRAVIS_BRANCH" = "master" ] && [ "$TRAVIS_PULL_REQUEST" = false ]; then /bin/bash -ex .travis/deploy-linux.bash ; fi
-53
View File
@@ -1,53 +0,0 @@
#!/bin/env bash -ex
shopt -s nocasematch
cd build
if [ "$DEPLOY_APPIMAGE" = "true" ]; then
DESTDIR=appdir ninja install
curl -sLO "https://github.com/probonopd/linuxdeployqt/releases/download/continuous/linuxdeployqt-continuous-x86_64.AppImage"
chmod a+x linuxdeployqt*.AppImage
./linuxdeployqt*.AppImage --appimage-extract
./squashfs-root/AppRun ./appdir/usr/share/applications/*.desktop -bundle-non-qt-libs
ls ./appdir/usr/lib/
rm -r ./appdir/usr/share/doc
rm ./appdir/usr/lib/libxcb*
cp $(readlink -f /lib/x86_64-linux-gnu/libnsl.so.1) ./appdir/usr/lib/libnsl.so.1
export PATH=/rpcs3/build/squashfs-root/usr/bin/:${PATH}
# Embed newer libstdc++ for distros that don't come with it (ubuntu 14.04, 16.04)
mkdir -p appdir/usr/optional/ ; mkdir -p appdir/usr/optional/libstdc++/
cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ./appdir/usr/optional/libstdc++/
rm ./appdir/AppRun
curl -sL https://github.com/RPCS3/AppImageKit-checkrt/releases/download/continuous2/AppRun-patched-x86_64 -o ./appdir/AppRun
chmod a+x ./appdir/AppRun
curl -sL https://github.com/RPCS3/AppImageKit-checkrt/releases/download/continuous2/exec-x86_64.so -o ./appdir/usr/optional/exec.so
# Compile checker binary for AppImageKit-checkrt
# This may need updating if you update the compiler or rpcs3 uses newer c++ features
# See https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/config/abi/pre/gnu.ver
# for which definitions correlate to which CXXABI version.
printf "#include <memory>\nint main(){std::make_exception_ptr(0);}" | $CXX -x c++ -o ./appdir/usr/optional/checker -
# Package it up and send it off
./squashfs-root/usr/bin/appimagetool /rpcs3/build/appdir
ls
COMM_TAG="$(git describe --tags $(git rev-list --tags --max-count=1))"
COMM_COUNT="$(git rev-list --count HEAD)"
curl -sLO https://github.com/hcorion/uploadtool/raw/master/upload.sh
mv ./RPCS3*.AppImage rpcs3-${COMM_TAG}-${COMM_COUNT}-${TRAVIS_COMMIT:0:8}_linux64.AppImage
FILESIZEMB=$(bc <<< "scale=6; $(stat -c %s ./rpcs3*.AppImage)/1000000")
SHA256SUM=($(sha256sum ./rpcs3*.AppImage))
unset TRAVIS_REPO_SLUG
REPO_SLUG=RPCS3/rpcs3-binaries-linux \
UPLOADTOOL_BODY="$SHA256SUM;${FILESIZEMB}MB"\
RELEASE_NAME=build-${TRAVIS_COMMIT}\
RELEASE_TITLE=${COMM_TAG}-${COMM_COUNT}\
REPO_COMMIT=d812f1254a1157c80fd402f94446310560f54e5f\
bash upload.sh rpcs3*.AppImage
fi
if [ "$DEPLOY_PPA" = "true" ]; then
export DEBFULLNAME="RPCS3 Build Bot"
fi
+202
View File
@@ -0,0 +1,202 @@
/* 7z.h -- 7z interface
2017-04-03 : Igor Pavlov : Public domain */
#ifndef __7Z_H
#define __7Z_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define k7zStartHeaderSize 0x20
#define k7zSignatureSize 6
extern const Byte k7zSignature[k7zSignatureSize];
typedef struct
{
const Byte *Data;
size_t Size;
} CSzData;
/* CSzCoderInfo & CSzFolder support only default methods */
typedef struct
{
size_t PropsOffset;
UInt32 MethodID;
Byte NumStreams;
Byte PropsSize;
} CSzCoderInfo;
typedef struct
{
UInt32 InIndex;
UInt32 OutIndex;
} CSzBond;
#define SZ_NUM_CODERS_IN_FOLDER_MAX 4
#define SZ_NUM_BONDS_IN_FOLDER_MAX 3
#define SZ_NUM_PACK_STREAMS_IN_FOLDER_MAX 4
typedef struct
{
UInt32 NumCoders;
UInt32 NumBonds;
UInt32 NumPackStreams;
UInt32 UnpackStream;
UInt32 PackStreams[SZ_NUM_PACK_STREAMS_IN_FOLDER_MAX];
CSzBond Bonds[SZ_NUM_BONDS_IN_FOLDER_MAX];
CSzCoderInfo Coders[SZ_NUM_CODERS_IN_FOLDER_MAX];
} CSzFolder;
SRes SzGetNextFolderItem(CSzFolder *f, CSzData *sd);
typedef struct
{
UInt32 Low;
UInt32 High;
} CNtfsFileTime;
typedef struct
{
Byte *Defs; /* MSB 0 bit numbering */
UInt32 *Vals;
} CSzBitUi32s;
typedef struct
{
Byte *Defs; /* MSB 0 bit numbering */
// UInt64 *Vals;
CNtfsFileTime *Vals;
} CSzBitUi64s;
#define SzBitArray_Check(p, i) (((p)[(i) >> 3] & (0x80 >> ((i) & 7))) != 0)
#define SzBitWithVals_Check(p, i) ((p)->Defs && ((p)->Defs[(i) >> 3] & (0x80 >> ((i) & 7))) != 0)
typedef struct
{
UInt32 NumPackStreams;
UInt32 NumFolders;
UInt64 *PackPositions; // NumPackStreams + 1
CSzBitUi32s FolderCRCs; // NumFolders
size_t *FoCodersOffsets; // NumFolders + 1
UInt32 *FoStartPackStreamIndex; // NumFolders + 1
UInt32 *FoToCoderUnpackSizes; // NumFolders + 1
Byte *FoToMainUnpackSizeIndex; // NumFolders
UInt64 *CoderUnpackSizes; // for all coders in all folders
Byte *CodersData;
} CSzAr;
UInt64 SzAr_GetFolderUnpackSize(const CSzAr *p, UInt32 folderIndex);
SRes SzAr_DecodeFolder(const CSzAr *p, UInt32 folderIndex,
ILookInStream *stream, UInt64 startPos,
Byte *outBuffer, size_t outSize,
ISzAllocPtr allocMain);
typedef struct
{
CSzAr db;
UInt64 startPosAfterHeader;
UInt64 dataPos;
UInt32 NumFiles;
UInt64 *UnpackPositions; // NumFiles + 1
// Byte *IsEmptyFiles;
Byte *IsDirs;
CSzBitUi32s CRCs;
CSzBitUi32s Attribs;
// CSzBitUi32s Parents;
CSzBitUi64s MTime;
CSzBitUi64s CTime;
UInt32 *FolderToFile; // NumFolders + 1
UInt32 *FileToFolder; // NumFiles
size_t *FileNameOffsets; /* in 2-byte steps */
Byte *FileNames; /* UTF-16-LE */
} CSzArEx;
#define SzArEx_IsDir(p, i) (SzBitArray_Check((p)->IsDirs, i))
#define SzArEx_GetFileSize(p, i) ((p)->UnpackPositions[(i) + 1] - (p)->UnpackPositions[i])
void SzArEx_Init(CSzArEx *p);
void SzArEx_Free(CSzArEx *p, ISzAllocPtr alloc);
UInt64 SzArEx_GetFolderStreamPos(const CSzArEx *p, UInt32 folderIndex, UInt32 indexInFolder);
int SzArEx_GetFolderFullPackSize(const CSzArEx *p, UInt32 folderIndex, UInt64 *resSize);
/*
if dest == NULL, the return value specifies the required size of the buffer,
in 16-bit characters, including the null-terminating character.
if dest != NULL, the return value specifies the number of 16-bit characters that
are written to the dest, including the null-terminating character. */
size_t SzArEx_GetFileNameUtf16(const CSzArEx *p, size_t fileIndex, UInt16 *dest);
/*
size_t SzArEx_GetFullNameLen(const CSzArEx *p, size_t fileIndex);
UInt16 *SzArEx_GetFullNameUtf16_Back(const CSzArEx *p, size_t fileIndex, UInt16 *dest);
*/
/*
SzArEx_Extract extracts file from archive
*outBuffer must be 0 before first call for each new archive.
Extracting cache:
If you need to decompress more than one file, you can send
these values from previous call:
*blockIndex,
*outBuffer,
*outBufferSize
You can consider "*outBuffer" as cache of solid block. If your archive is solid,
it will increase decompression speed.
If you use external function, you can declare these 3 cache variables
(blockIndex, outBuffer, outBufferSize) as static in that external function.
Free *outBuffer and set *outBuffer to 0, if you want to flush cache.
*/
SRes SzArEx_Extract(
const CSzArEx *db,
ILookInStream *inStream,
UInt32 fileIndex, /* index of file */
UInt32 *blockIndex, /* index of solid block */
Byte **outBuffer, /* pointer to pointer to output buffer (allocated with allocMain) */
size_t *outBufferSize, /* buffer size for output buffer */
size_t *offset, /* offset of stream for required file in *outBuffer */
size_t *outSizeProcessed, /* size of file in *outBuffer */
ISzAllocPtr allocMain,
ISzAllocPtr allocTemp);
/*
SzArEx_Open Errors:
SZ_ERROR_NO_ARCHIVE
SZ_ERROR_ARCHIVE
SZ_ERROR_UNSUPPORTED
SZ_ERROR_MEM
SZ_ERROR_CRC
SZ_ERROR_INPUT_EOF
SZ_ERROR_FAIL
*/
SRes SzArEx_Open(CSzArEx *p, ILookInStream *inStream,
ISzAllocPtr allocMain, ISzAllocPtr allocTemp);
EXTERN_C_END
#endif
+80
View File
@@ -0,0 +1,80 @@
/* 7zAlloc.c -- Allocation functions
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <stdlib.h>
#include "7zAlloc.h"
/* #define _SZ_ALLOC_DEBUG */
/* use _SZ_ALLOC_DEBUG to debug alloc/free operations */
#ifdef _SZ_ALLOC_DEBUG
#ifdef _WIN32
#include <windows.h>
#endif
#include <stdio.h>
int g_allocCount = 0;
int g_allocCountTemp = 0;
#endif
void *SzAlloc(ISzAllocPtr p, size_t size)
{
UNUSED_VAR(p);
if (size == 0)
return 0;
#ifdef _SZ_ALLOC_DEBUG
fprintf(stderr, "\nAlloc %10u bytes; count = %10d", (unsigned)size, g_allocCount);
g_allocCount++;
#endif
return malloc(size);
}
void SzFree(ISzAllocPtr p, void *address)
{
UNUSED_VAR(p);
#ifdef _SZ_ALLOC_DEBUG
if (address != 0)
{
g_allocCount--;
fprintf(stderr, "\nFree; count = %10d", g_allocCount);
}
#endif
free(address);
}
void *SzAllocTemp(ISzAllocPtr p, size_t size)
{
UNUSED_VAR(p);
if (size == 0)
return 0;
#ifdef _SZ_ALLOC_DEBUG
fprintf(stderr, "\nAlloc_temp %10u bytes; count = %10d", (unsigned)size, g_allocCountTemp);
g_allocCountTemp++;
#ifdef _WIN32
return HeapAlloc(GetProcessHeap(), 0, size);
#endif
#endif
return malloc(size);
}
void SzFreeTemp(ISzAllocPtr p, void *address)
{
UNUSED_VAR(p);
#ifdef _SZ_ALLOC_DEBUG
if (address != 0)
{
g_allocCountTemp--;
fprintf(stderr, "\nFree_temp; count = %10d", g_allocCountTemp);
}
#ifdef _WIN32
HeapFree(GetProcessHeap(), 0, address);
return;
#endif
#endif
free(address);
}
+19
View File
@@ -0,0 +1,19 @@
/* 7zAlloc.h -- Allocation functions
2017-04-03 : Igor Pavlov : Public domain */
#ifndef __7Z_ALLOC_H
#define __7Z_ALLOC_H
#include "7zTypes.h"
EXTERN_C_BEGIN
void *SzAlloc(ISzAllocPtr p, size_t size);
void SzFree(ISzAllocPtr p, void *address);
void *SzAllocTemp(ISzAllocPtr p, size_t size);
void SzFreeTemp(ISzAllocPtr p, void *address);
EXTERN_C_END
#endif
+1771
View File
File diff suppressed because it is too large Load Diff
+36
View File
@@ -0,0 +1,36 @@
/* 7zBuf.c -- Byte Buffer
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "7zBuf.h"
void Buf_Init(CBuf *p)
{
p->data = 0;
p->size = 0;
}
int Buf_Create(CBuf *p, size_t size, ISzAllocPtr alloc)
{
p->size = 0;
if (size == 0)
{
p->data = 0;
return 1;
}
p->data = (Byte *)ISzAlloc_Alloc(alloc, size);
if (p->data)
{
p->size = size;
return 1;
}
return 0;
}
void Buf_Free(CBuf *p, ISzAllocPtr alloc)
{
ISzAlloc_Free(alloc, p->data);
p->data = 0;
p->size = 0;
}
+35
View File
@@ -0,0 +1,35 @@
/* 7zBuf.h -- Byte Buffer
2017-04-03 : Igor Pavlov : Public domain */
#ifndef __7Z_BUF_H
#define __7Z_BUF_H
#include "7zTypes.h"
EXTERN_C_BEGIN
typedef struct
{
Byte *data;
size_t size;
} CBuf;
void Buf_Init(CBuf *p);
int Buf_Create(CBuf *p, size_t size, ISzAllocPtr alloc);
void Buf_Free(CBuf *p, ISzAllocPtr alloc);
typedef struct
{
Byte *data;
size_t size;
size_t pos;
} CDynBuf;
void DynBuf_Construct(CDynBuf *p);
void DynBuf_SeekToBeg(CDynBuf *p);
int DynBuf_Write(CDynBuf *p, const Byte *buf, size_t size, ISzAllocPtr alloc);
void DynBuf_Free(CDynBuf *p, ISzAllocPtr alloc);
EXTERN_C_END
#endif
+52
View File
@@ -0,0 +1,52 @@
/* 7zBuf2.c -- Byte Buffer
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <string.h>
#include "7zBuf.h"
void DynBuf_Construct(CDynBuf *p)
{
p->data = 0;
p->size = 0;
p->pos = 0;
}
void DynBuf_SeekToBeg(CDynBuf *p)
{
p->pos = 0;
}
int DynBuf_Write(CDynBuf *p, const Byte *buf, size_t size, ISzAllocPtr alloc)
{
if (size > p->size - p->pos)
{
size_t newSize = p->pos + size;
Byte *data;
newSize += newSize / 4;
data = (Byte *)ISzAlloc_Alloc(alloc, newSize);
if (!data)
return 0;
p->size = newSize;
if (p->pos != 0)
memcpy(data, p->data, p->pos);
ISzAlloc_Free(alloc, p->data);
p->data = data;
}
if (size != 0)
{
memcpy(p->data + p->pos, buf, size);
p->pos += size;
}
return 1;
}
void DynBuf_Free(CDynBuf *p, ISzAllocPtr alloc)
{
ISzAlloc_Free(alloc, p->data);
p->data = 0;
p->size = 0;
p->pos = 0;
}
+128
View File
@@ -0,0 +1,128 @@
/* 7zCrc.c -- CRC32 init
2017-06-06 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "7zCrc.h"
#include "CpuArch.h"
#define kCrcPoly 0xEDB88320
#ifdef MY_CPU_LE
#define CRC_NUM_TABLES 8
#else
#define CRC_NUM_TABLES 9
#define CRC_UINT32_SWAP(v) ((v >> 24) | ((v >> 8) & 0xFF00) | ((v << 8) & 0xFF0000) | (v << 24))
UInt32 MY_FAST_CALL CrcUpdateT1_BeT4(UInt32 v, const void *data, size_t size, const UInt32 *table);
UInt32 MY_FAST_CALL CrcUpdateT1_BeT8(UInt32 v, const void *data, size_t size, const UInt32 *table);
#endif
#ifndef MY_CPU_BE
UInt32 MY_FAST_CALL CrcUpdateT4(UInt32 v, const void *data, size_t size, const UInt32 *table);
UInt32 MY_FAST_CALL CrcUpdateT8(UInt32 v, const void *data, size_t size, const UInt32 *table);
#endif
typedef UInt32 (MY_FAST_CALL *CRC_FUNC)(UInt32 v, const void *data, size_t size, const UInt32 *table);
CRC_FUNC g_CrcUpdateT4;
CRC_FUNC g_CrcUpdateT8;
CRC_FUNC g_CrcUpdate;
UInt32 g_CrcTable[256 * CRC_NUM_TABLES];
UInt32 MY_FAST_CALL CrcUpdate(UInt32 v, const void *data, size_t size)
{
return g_CrcUpdate(v, data, size, g_CrcTable);
}
UInt32 MY_FAST_CALL CrcCalc(const void *data, size_t size)
{
return g_CrcUpdate(CRC_INIT_VAL, data, size, g_CrcTable) ^ CRC_INIT_VAL;
}
#define CRC_UPDATE_BYTE_2(crc, b) (table[((crc) ^ (b)) & 0xFF] ^ ((crc) >> 8))
UInt32 MY_FAST_CALL CrcUpdateT1(UInt32 v, const void *data, size_t size, const UInt32 *table)
{
const Byte *p = (const Byte *)data;
const Byte *pEnd = p + size;
for (; p != pEnd; p++)
v = CRC_UPDATE_BYTE_2(v, *p);
return v;
}
void MY_FAST_CALL CrcGenerateTable()
{
UInt32 i;
for (i = 0; i < 256; i++)
{
UInt32 r = i;
unsigned j;
for (j = 0; j < 8; j++)
r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
g_CrcTable[i] = r;
}
for (i = 256; i < 256 * CRC_NUM_TABLES; i++)
{
UInt32 r = g_CrcTable[(size_t)i - 256];
g_CrcTable[i] = g_CrcTable[r & 0xFF] ^ (r >> 8);
}
#if CRC_NUM_TABLES < 4
g_CrcUpdate = CrcUpdateT1;
#else
#ifdef MY_CPU_LE
g_CrcUpdateT4 = CrcUpdateT4;
g_CrcUpdate = CrcUpdateT4;
#if CRC_NUM_TABLES >= 8
g_CrcUpdateT8 = CrcUpdateT8;
#ifdef MY_CPU_X86_OR_AMD64
if (!CPU_Is_InOrder())
#endif
g_CrcUpdate = CrcUpdateT8;
#endif
#else
{
#ifndef MY_CPU_BE
UInt32 k = 0x01020304;
const Byte *p = (const Byte *)&k;
if (p[0] == 4 && p[1] == 3)
{
g_CrcUpdateT4 = CrcUpdateT4;
g_CrcUpdate = CrcUpdateT4;
#if CRC_NUM_TABLES >= 8
g_CrcUpdateT8 = CrcUpdateT8;
g_CrcUpdate = CrcUpdateT8;
#endif
}
else if (p[0] != 1 || p[1] != 2)
g_CrcUpdate = CrcUpdateT1;
else
#endif
{
for (i = 256 * CRC_NUM_TABLES - 1; i >= 256; i--)
{
UInt32 x = g_CrcTable[(size_t)i - 256];
g_CrcTable[i] = CRC_UINT32_SWAP(x);
}
g_CrcUpdateT4 = CrcUpdateT1_BeT4;
g_CrcUpdate = CrcUpdateT1_BeT4;
#if CRC_NUM_TABLES >= 8
g_CrcUpdateT8 = CrcUpdateT1_BeT8;
g_CrcUpdate = CrcUpdateT1_BeT8;
#endif
}
}
#endif
#endif
}
+25
View File
@@ -0,0 +1,25 @@
/* 7zCrc.h -- CRC32 calculation
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __7Z_CRC_H
#define __7Z_CRC_H
#include "7zTypes.h"
EXTERN_C_BEGIN
extern UInt32 g_CrcTable[];
/* Call CrcGenerateTable one time before other CRC functions */
void MY_FAST_CALL CrcGenerateTable(void);
#define CRC_INIT_VAL 0xFFFFFFFF
#define CRC_GET_DIGEST(crc) ((crc) ^ CRC_INIT_VAL)
#define CRC_UPDATE_BYTE(crc, b) (g_CrcTable[((crc) ^ (b)) & 0xFF] ^ ((crc) >> 8))
UInt32 MY_FAST_CALL CrcUpdate(UInt32 crc, const void *data, size_t size);
UInt32 MY_FAST_CALL CrcCalc(const void *data, size_t size);
EXTERN_C_END
#endif
+115
View File
@@ -0,0 +1,115 @@
/* 7zCrcOpt.c -- CRC32 calculation
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "CpuArch.h"
#ifndef MY_CPU_BE
#define CRC_UPDATE_BYTE_2(crc, b) (table[((crc) ^ (b)) & 0xFF] ^ ((crc) >> 8))
UInt32 MY_FAST_CALL CrcUpdateT4(UInt32 v, const void *data, size_t size, const UInt32 *table)
{
const Byte *p = (const Byte *)data;
for (; size > 0 && ((unsigned)(ptrdiff_t)p & 3) != 0; size--, p++)
v = CRC_UPDATE_BYTE_2(v, *p);
for (; size >= 4; size -= 4, p += 4)
{
v ^= *(const UInt32 *)p;
v =
(table + 0x300)[((v ) & 0xFF)]
^ (table + 0x200)[((v >> 8) & 0xFF)]
^ (table + 0x100)[((v >> 16) & 0xFF)]
^ (table + 0x000)[((v >> 24))];
}
for (; size > 0; size--, p++)
v = CRC_UPDATE_BYTE_2(v, *p);
return v;
}
UInt32 MY_FAST_CALL CrcUpdateT8(UInt32 v, const void *data, size_t size, const UInt32 *table)
{
const Byte *p = (const Byte *)data;
for (; size > 0 && ((unsigned)(ptrdiff_t)p & 7) != 0; size--, p++)
v = CRC_UPDATE_BYTE_2(v, *p);
for (; size >= 8; size -= 8, p += 8)
{
UInt32 d;
v ^= *(const UInt32 *)p;
v =
(table + 0x700)[((v ) & 0xFF)]
^ (table + 0x600)[((v >> 8) & 0xFF)]
^ (table + 0x500)[((v >> 16) & 0xFF)]
^ (table + 0x400)[((v >> 24))];
d = *((const UInt32 *)p + 1);
v ^=
(table + 0x300)[((d ) & 0xFF)]
^ (table + 0x200)[((d >> 8) & 0xFF)]
^ (table + 0x100)[((d >> 16) & 0xFF)]
^ (table + 0x000)[((d >> 24))];
}
for (; size > 0; size--, p++)
v = CRC_UPDATE_BYTE_2(v, *p);
return v;
}
#endif
#ifndef MY_CPU_LE
#define CRC_UINT32_SWAP(v) ((v >> 24) | ((v >> 8) & 0xFF00) | ((v << 8) & 0xFF0000) | (v << 24))
#define CRC_UPDATE_BYTE_2_BE(crc, b) (table[(((crc) >> 24) ^ (b))] ^ ((crc) << 8))
UInt32 MY_FAST_CALL CrcUpdateT1_BeT4(UInt32 v, const void *data, size_t size, const UInt32 *table)
{
const Byte *p = (const Byte *)data;
table += 0x100;
v = CRC_UINT32_SWAP(v);
for (; size > 0 && ((unsigned)(ptrdiff_t)p & 3) != 0; size--, p++)
v = CRC_UPDATE_BYTE_2_BE(v, *p);
for (; size >= 4; size -= 4, p += 4)
{
v ^= *(const UInt32 *)p;
v =
(table + 0x000)[((v ) & 0xFF)]
^ (table + 0x100)[((v >> 8) & 0xFF)]
^ (table + 0x200)[((v >> 16) & 0xFF)]
^ (table + 0x300)[((v >> 24))];
}
for (; size > 0; size--, p++)
v = CRC_UPDATE_BYTE_2_BE(v, *p);
return CRC_UINT32_SWAP(v);
}
UInt32 MY_FAST_CALL CrcUpdateT1_BeT8(UInt32 v, const void *data, size_t size, const UInt32 *table)
{
const Byte *p = (const Byte *)data;
table += 0x100;
v = CRC_UINT32_SWAP(v);
for (; size > 0 && ((unsigned)(ptrdiff_t)p & 7) != 0; size--, p++)
v = CRC_UPDATE_BYTE_2_BE(v, *p);
for (; size >= 8; size -= 8, p += 8)
{
UInt32 d;
v ^= *(const UInt32 *)p;
v =
(table + 0x400)[((v ) & 0xFF)]
^ (table + 0x500)[((v >> 8) & 0xFF)]
^ (table + 0x600)[((v >> 16) & 0xFF)]
^ (table + 0x700)[((v >> 24))];
d = *((const UInt32 *)p + 1);
v ^=
(table + 0x000)[((d ) & 0xFF)]
^ (table + 0x100)[((d >> 8) & 0xFF)]
^ (table + 0x200)[((d >> 16) & 0xFF)]
^ (table + 0x300)[((d >> 24))];
}
for (; size > 0; size--, p++)
v = CRC_UPDATE_BYTE_2_BE(v, *p);
return CRC_UINT32_SWAP(v);
}
#endif
+591
View File
@@ -0,0 +1,591 @@
/* 7zDec.c -- Decoding from 7z folder
2019-02-02 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <string.h>
/* #define _7ZIP_PPMD_SUPPPORT */
#include "7z.h"
#include "7zCrc.h"
#include "Bcj2.h"
#include "Bra.h"
#include "CpuArch.h"
#include "Delta.h"
#include "LzmaDec.h"
#include "Lzma2Dec.h"
#ifdef _7ZIP_PPMD_SUPPPORT
#include "Ppmd7.h"
#endif
#define k_Copy 0
#define k_Delta 3
#define k_LZMA2 0x21
#define k_LZMA 0x30101
#define k_BCJ 0x3030103
#define k_BCJ2 0x303011B
#define k_PPC 0x3030205
#define k_IA64 0x3030401
#define k_ARM 0x3030501
#define k_ARMT 0x3030701
#define k_SPARC 0x3030805
#ifdef _7ZIP_PPMD_SUPPPORT
#define k_PPMD 0x30401
typedef struct
{
IByteIn vt;
const Byte *cur;
const Byte *end;
const Byte *begin;
UInt64 processed;
BoolInt extra;
SRes res;
const ILookInStream *inStream;
} CByteInToLook;
static Byte ReadByte(const IByteIn *pp)
{
CByteInToLook *p = CONTAINER_FROM_VTBL(pp, CByteInToLook, vt);
if (p->cur != p->end)
return *p->cur++;
if (p->res == SZ_OK)
{
size_t size = p->cur - p->begin;
p->processed += size;
p->res = ILookInStream_Skip(p->inStream, size);
size = (1 << 25);
p->res = ILookInStream_Look(p->inStream, (const void **)&p->begin, &size);
p->cur = p->begin;
p->end = p->begin + size;
if (size != 0)
return *p->cur++;;
}
p->extra = True;
return 0;
}
static SRes SzDecodePpmd(const Byte *props, unsigned propsSize, UInt64 inSize, const ILookInStream *inStream,
Byte *outBuffer, SizeT outSize, ISzAllocPtr allocMain)
{
CPpmd7 ppmd;
CByteInToLook s;
SRes res = SZ_OK;
s.vt.Read = ReadByte;
s.inStream = inStream;
s.begin = s.end = s.cur = NULL;
s.extra = False;
s.res = SZ_OK;
s.processed = 0;
if (propsSize != 5)
return SZ_ERROR_UNSUPPORTED;
{
unsigned order = props[0];
UInt32 memSize = GetUi32(props + 1);
if (order < PPMD7_MIN_ORDER ||
order > PPMD7_MAX_ORDER ||
memSize < PPMD7_MIN_MEM_SIZE ||
memSize > PPMD7_MAX_MEM_SIZE)
return SZ_ERROR_UNSUPPORTED;
Ppmd7_Construct(&ppmd);
if (!Ppmd7_Alloc(&ppmd, memSize, allocMain))
return SZ_ERROR_MEM;
Ppmd7_Init(&ppmd, order);
}
{
CPpmd7z_RangeDec rc;
Ppmd7z_RangeDec_CreateVTable(&rc);
rc.Stream = &s.vt;
if (!Ppmd7z_RangeDec_Init(&rc))
res = SZ_ERROR_DATA;
else if (s.extra)
res = (s.res != SZ_OK ? s.res : SZ_ERROR_DATA);
else
{
SizeT i;
for (i = 0; i < outSize; i++)
{
int sym = Ppmd7_DecodeSymbol(&ppmd, &rc.vt);
if (s.extra || sym < 0)
break;
outBuffer[i] = (Byte)sym;
}
if (i != outSize)
res = (s.res != SZ_OK ? s.res : SZ_ERROR_DATA);
else if (s.processed + (s.cur - s.begin) != inSize || !Ppmd7z_RangeDec_IsFinishedOK(&rc))
res = SZ_ERROR_DATA;
}
}
Ppmd7_Free(&ppmd, allocMain);
return res;
}
#endif
static SRes SzDecodeLzma(const Byte *props, unsigned propsSize, UInt64 inSize, ILookInStream *inStream,
Byte *outBuffer, SizeT outSize, ISzAllocPtr allocMain)
{
CLzmaDec state;
SRes res = SZ_OK;
LzmaDec_Construct(&state);
RINOK(LzmaDec_AllocateProbs(&state, props, propsSize, allocMain));
state.dic = outBuffer;
state.dicBufSize = outSize;
LzmaDec_Init(&state);
for (;;)
{
const void *inBuf = NULL;
size_t lookahead = (1 << 18);
if (lookahead > inSize)
lookahead = (size_t)inSize;
res = ILookInStream_Look(inStream, &inBuf, &lookahead);
if (res != SZ_OK)
break;
{
SizeT inProcessed = (SizeT)lookahead, dicPos = state.dicPos;
ELzmaStatus status;
res = LzmaDec_DecodeToDic(&state, outSize, (const Byte *)inBuf, &inProcessed, LZMA_FINISH_END, &status);
lookahead -= inProcessed;
inSize -= inProcessed;
if (res != SZ_OK)
break;
if (status == LZMA_STATUS_FINISHED_WITH_MARK)
{
if (outSize != state.dicPos || inSize != 0)
res = SZ_ERROR_DATA;
break;
}
if (outSize == state.dicPos && inSize == 0 && status == LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK)
break;
if (inProcessed == 0 && dicPos == state.dicPos)
{
res = SZ_ERROR_DATA;
break;
}
res = ILookInStream_Skip(inStream, inProcessed);
if (res != SZ_OK)
break;
}
}
LzmaDec_FreeProbs(&state, allocMain);
return res;
}
#ifndef _7Z_NO_METHOD_LZMA2
static SRes SzDecodeLzma2(const Byte *props, unsigned propsSize, UInt64 inSize, ILookInStream *inStream,
Byte *outBuffer, SizeT outSize, ISzAllocPtr allocMain)
{
CLzma2Dec state;
SRes res = SZ_OK;
Lzma2Dec_Construct(&state);
if (propsSize != 1)
return SZ_ERROR_DATA;
RINOK(Lzma2Dec_AllocateProbs(&state, props[0], allocMain));
state.decoder.dic = outBuffer;
state.decoder.dicBufSize = outSize;
Lzma2Dec_Init(&state);
for (;;)
{
const void *inBuf = NULL;
size_t lookahead = (1 << 18);
if (lookahead > inSize)
lookahead = (size_t)inSize;
res = ILookInStream_Look(inStream, &inBuf, &lookahead);
if (res != SZ_OK)
break;
{
SizeT inProcessed = (SizeT)lookahead, dicPos = state.decoder.dicPos;
ELzmaStatus status;
res = Lzma2Dec_DecodeToDic(&state, outSize, (const Byte *)inBuf, &inProcessed, LZMA_FINISH_END, &status);
lookahead -= inProcessed;
inSize -= inProcessed;
if (res != SZ_OK)
break;
if (status == LZMA_STATUS_FINISHED_WITH_MARK)
{
if (outSize != state.decoder.dicPos || inSize != 0)
res = SZ_ERROR_DATA;
break;
}
if (inProcessed == 0 && dicPos == state.decoder.dicPos)
{
res = SZ_ERROR_DATA;
break;
}
res = ILookInStream_Skip(inStream, inProcessed);
if (res != SZ_OK)
break;
}
}
Lzma2Dec_FreeProbs(&state, allocMain);
return res;
}
#endif
static SRes SzDecodeCopy(UInt64 inSize, ILookInStream *inStream, Byte *outBuffer)
{
while (inSize > 0)
{
const void *inBuf;
size_t curSize = (1 << 18);
if (curSize > inSize)
curSize = (size_t)inSize;
RINOK(ILookInStream_Look(inStream, &inBuf, &curSize));
if (curSize == 0)
return SZ_ERROR_INPUT_EOF;
memcpy(outBuffer, inBuf, curSize);
outBuffer += curSize;
inSize -= curSize;
RINOK(ILookInStream_Skip(inStream, curSize));
}
return SZ_OK;
}
static BoolInt IS_MAIN_METHOD(UInt32 m)
{
switch (m)
{
case k_Copy:
case k_LZMA:
#ifndef _7Z_NO_METHOD_LZMA2
case k_LZMA2:
#endif
#ifdef _7ZIP_PPMD_SUPPPORT
case k_PPMD:
#endif
return True;
}
return False;
}
static BoolInt IS_SUPPORTED_CODER(const CSzCoderInfo *c)
{
return
c->NumStreams == 1
/* && c->MethodID <= (UInt32)0xFFFFFFFF */
&& IS_MAIN_METHOD((UInt32)c->MethodID);
}
#define IS_BCJ2(c) ((c)->MethodID == k_BCJ2 && (c)->NumStreams == 4)
static SRes CheckSupportedFolder(const CSzFolder *f)
{
if (f->NumCoders < 1 || f->NumCoders > 4)
return SZ_ERROR_UNSUPPORTED;
if (!IS_SUPPORTED_CODER(&f->Coders[0]))
return SZ_ERROR_UNSUPPORTED;
if (f->NumCoders == 1)
{
if (f->NumPackStreams != 1 || f->PackStreams[0] != 0 || f->NumBonds != 0)
return SZ_ERROR_UNSUPPORTED;
return SZ_OK;
}
#ifndef _7Z_NO_METHODS_FILTERS
if (f->NumCoders == 2)
{
const CSzCoderInfo *c = &f->Coders[1];
if (
/* c->MethodID > (UInt32)0xFFFFFFFF || */
c->NumStreams != 1
|| f->NumPackStreams != 1
|| f->PackStreams[0] != 0
|| f->NumBonds != 1
|| f->Bonds[0].InIndex != 1
|| f->Bonds[0].OutIndex != 0)
return SZ_ERROR_UNSUPPORTED;
switch ((UInt32)c->MethodID)
{
case k_Delta:
case k_BCJ:
case k_PPC:
case k_IA64:
case k_SPARC:
case k_ARM:
case k_ARMT:
break;
default:
return SZ_ERROR_UNSUPPORTED;
}
return SZ_OK;
}
#endif
if (f->NumCoders == 4)
{
if (!IS_SUPPORTED_CODER(&f->Coders[1])
|| !IS_SUPPORTED_CODER(&f->Coders[2])
|| !IS_BCJ2(&f->Coders[3]))
return SZ_ERROR_UNSUPPORTED;
if (f->NumPackStreams != 4
|| f->PackStreams[0] != 2
|| f->PackStreams[1] != 6
|| f->PackStreams[2] != 1
|| f->PackStreams[3] != 0
|| f->NumBonds != 3
|| f->Bonds[0].InIndex != 5 || f->Bonds[0].OutIndex != 0
|| f->Bonds[1].InIndex != 4 || f->Bonds[1].OutIndex != 1
|| f->Bonds[2].InIndex != 3 || f->Bonds[2].OutIndex != 2)
return SZ_ERROR_UNSUPPORTED;
return SZ_OK;
}
return SZ_ERROR_UNSUPPORTED;
}
#define CASE_BRA_CONV(isa) case k_ ## isa: isa ## _Convert(outBuffer, outSize, 0, 0); break;
static SRes SzFolder_Decode2(const CSzFolder *folder,
const Byte *propsData,
const UInt64 *unpackSizes,
const UInt64 *packPositions,
ILookInStream *inStream, UInt64 startPos,
Byte *outBuffer, SizeT outSize, ISzAllocPtr allocMain,
Byte *tempBuf[])
{
UInt32 ci;
SizeT tempSizes[3] = { 0, 0, 0};
SizeT tempSize3 = 0;
Byte *tempBuf3 = 0;
RINOK(CheckSupportedFolder(folder));
for (ci = 0; ci < folder->NumCoders; ci++)
{
const CSzCoderInfo *coder = &folder->Coders[ci];
if (IS_MAIN_METHOD((UInt32)coder->MethodID))
{
UInt32 si = 0;
UInt64 offset;
UInt64 inSize;
Byte *outBufCur = outBuffer;
SizeT outSizeCur = outSize;
if (folder->NumCoders == 4)
{
UInt32 indices[] = { 3, 2, 0 };
UInt64 unpackSize = unpackSizes[ci];
si = indices[ci];
if (ci < 2)
{
Byte *temp;
outSizeCur = (SizeT)unpackSize;
if (outSizeCur != unpackSize)
return SZ_ERROR_MEM;
temp = (Byte *)ISzAlloc_Alloc(allocMain, outSizeCur);
if (!temp && outSizeCur != 0)
return SZ_ERROR_MEM;
outBufCur = tempBuf[1 - ci] = temp;
tempSizes[1 - ci] = outSizeCur;
}
else if (ci == 2)
{
if (unpackSize > outSize) /* check it */
return SZ_ERROR_PARAM;
tempBuf3 = outBufCur = outBuffer + (outSize - (size_t)unpackSize);
tempSize3 = outSizeCur = (SizeT)unpackSize;
}
else
return SZ_ERROR_UNSUPPORTED;
}
offset = packPositions[si];
inSize = packPositions[(size_t)si + 1] - offset;
RINOK(LookInStream_SeekTo(inStream, startPos + offset));
if (coder->MethodID == k_Copy)
{
if (inSize != outSizeCur) /* check it */
return SZ_ERROR_DATA;
RINOK(SzDecodeCopy(inSize, inStream, outBufCur));
}
else if (coder->MethodID == k_LZMA)
{
RINOK(SzDecodeLzma(propsData + coder->PropsOffset, coder->PropsSize, inSize, inStream, outBufCur, outSizeCur, allocMain));
}
#ifndef _7Z_NO_METHOD_LZMA2
else if (coder->MethodID == k_LZMA2)
{
RINOK(SzDecodeLzma2(propsData + coder->PropsOffset, coder->PropsSize, inSize, inStream, outBufCur, outSizeCur, allocMain));
}
#endif
#ifdef _7ZIP_PPMD_SUPPPORT
else if (coder->MethodID == k_PPMD)
{
RINOK(SzDecodePpmd(propsData + coder->PropsOffset, coder->PropsSize, inSize, inStream, outBufCur, outSizeCur, allocMain));
}
#endif
else
return SZ_ERROR_UNSUPPORTED;
}
else if (coder->MethodID == k_BCJ2)
{
UInt64 offset = packPositions[1];
UInt64 s3Size = packPositions[2] - offset;
if (ci != 3)
return SZ_ERROR_UNSUPPORTED;
tempSizes[2] = (SizeT)s3Size;
if (tempSizes[2] != s3Size)
return SZ_ERROR_MEM;
tempBuf[2] = (Byte *)ISzAlloc_Alloc(allocMain, tempSizes[2]);
if (!tempBuf[2] && tempSizes[2] != 0)
return SZ_ERROR_MEM;
RINOK(LookInStream_SeekTo(inStream, startPos + offset));
RINOK(SzDecodeCopy(s3Size, inStream, tempBuf[2]));
if ((tempSizes[0] & 3) != 0 ||
(tempSizes[1] & 3) != 0 ||
tempSize3 + tempSizes[0] + tempSizes[1] != outSize)
return SZ_ERROR_DATA;
{
CBcj2Dec p;
p.bufs[0] = tempBuf3; p.lims[0] = tempBuf3 + tempSize3;
p.bufs[1] = tempBuf[0]; p.lims[1] = tempBuf[0] + tempSizes[0];
p.bufs[2] = tempBuf[1]; p.lims[2] = tempBuf[1] + tempSizes[1];
p.bufs[3] = tempBuf[2]; p.lims[3] = tempBuf[2] + tempSizes[2];
p.dest = outBuffer;
p.destLim = outBuffer + outSize;
Bcj2Dec_Init(&p);
RINOK(Bcj2Dec_Decode(&p));
{
unsigned i;
for (i = 0; i < 4; i++)
if (p.bufs[i] != p.lims[i])
return SZ_ERROR_DATA;
if (!Bcj2Dec_IsFinished(&p))
return SZ_ERROR_DATA;
if (p.dest != p.destLim
|| p.state != BCJ2_STREAM_MAIN)
return SZ_ERROR_DATA;
}
}
}
#ifndef _7Z_NO_METHODS_FILTERS
else if (ci == 1)
{
if (coder->MethodID == k_Delta)
{
if (coder->PropsSize != 1)
return SZ_ERROR_UNSUPPORTED;
{
Byte state[DELTA_STATE_SIZE];
Delta_Init(state);
Delta_Decode(state, (unsigned)(propsData[coder->PropsOffset]) + 1, outBuffer, outSize);
}
}
else
{
if (coder->PropsSize != 0)
return SZ_ERROR_UNSUPPORTED;
switch (coder->MethodID)
{
case k_BCJ:
{
UInt32 state;
x86_Convert_Init(state);
x86_Convert(outBuffer, outSize, 0, &state, 0);
break;
}
CASE_BRA_CONV(PPC)
CASE_BRA_CONV(IA64)
CASE_BRA_CONV(SPARC)
CASE_BRA_CONV(ARM)
CASE_BRA_CONV(ARMT)
default:
return SZ_ERROR_UNSUPPORTED;
}
}
}
#endif
else
return SZ_ERROR_UNSUPPORTED;
}
return SZ_OK;
}
SRes SzAr_DecodeFolder(const CSzAr *p, UInt32 folderIndex,
ILookInStream *inStream, UInt64 startPos,
Byte *outBuffer, size_t outSize,
ISzAllocPtr allocMain)
{
SRes res;
CSzFolder folder;
CSzData sd;
const Byte *data = p->CodersData + p->FoCodersOffsets[folderIndex];
sd.Data = data;
sd.Size = p->FoCodersOffsets[(size_t)folderIndex + 1] - p->FoCodersOffsets[folderIndex];
res = SzGetNextFolderItem(&folder, &sd);
if (res != SZ_OK)
return res;
if (sd.Size != 0
|| folder.UnpackStream != p->FoToMainUnpackSizeIndex[folderIndex]
|| outSize != SzAr_GetFolderUnpackSize(p, folderIndex))
return SZ_ERROR_FAIL;
{
unsigned i;
Byte *tempBuf[3] = { 0, 0, 0};
res = SzFolder_Decode2(&folder, data,
&p->CoderUnpackSizes[p->FoToCoderUnpackSizes[folderIndex]],
p->PackPositions + p->FoStartPackStreamIndex[folderIndex],
inStream, startPos,
outBuffer, (SizeT)outSize, allocMain, tempBuf);
for (i = 0; i < 3; i++)
ISzAlloc_Free(allocMain, tempBuf[i]);
if (res == SZ_OK)
if (SzBitWithVals_Check(&p->FolderCRCs, folderIndex))
if (CrcCalc(outBuffer, outSize) != p->FolderCRCs.Vals[folderIndex])
res = SZ_ERROR_CRC;
return res;
}
}
+286
View File
@@ -0,0 +1,286 @@
/* 7zFile.c -- File IO
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "7zFile.h"
#ifndef USE_WINDOWS_FILE
#ifndef UNDER_CE
#include <errno.h>
#endif
#else
/*
ReadFile and WriteFile functions in Windows have BUG:
If you Read or Write 64MB or more (probably min_failure_size = 64MB - 32KB + 1)
from/to Network file, it returns ERROR_NO_SYSTEM_RESOURCES
(Insufficient system resources exist to complete the requested service).
Probably in some version of Windows there are problems with other sizes:
for 32 MB (maybe also for 16 MB).
And message can be "Network connection was lost"
*/
#define kChunkSizeMax (1 << 22)
#endif
void File_Construct(CSzFile *p)
{
#ifdef USE_WINDOWS_FILE
p->handle = INVALID_HANDLE_VALUE;
#else
p->file = NULL;
#endif
}
#if !defined(UNDER_CE) || !defined(USE_WINDOWS_FILE)
static WRes File_Open(CSzFile *p, const char *name, int writeMode)
{
#ifdef USE_WINDOWS_FILE
p->handle = CreateFileA(name,
writeMode ? GENERIC_WRITE : GENERIC_READ,
FILE_SHARE_READ, NULL,
writeMode ? CREATE_ALWAYS : OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, NULL);
return (p->handle != INVALID_HANDLE_VALUE) ? 0 : GetLastError();
#else
p->file = fopen(name, writeMode ? "wb+" : "rb");
return (p->file != 0) ? 0 :
#ifdef UNDER_CE
2; /* ENOENT */
#else
errno;
#endif
#endif
}
WRes InFile_Open(CSzFile *p, const char *name) { return File_Open(p, name, 0); }
WRes OutFile_Open(CSzFile *p, const char *name) { return File_Open(p, name, 1); }
#endif
#ifdef USE_WINDOWS_FILE
static WRes File_OpenW(CSzFile *p, const WCHAR *name, int writeMode)
{
p->handle = CreateFileW(name,
writeMode ? GENERIC_WRITE : GENERIC_READ,
FILE_SHARE_READ, NULL,
writeMode ? CREATE_ALWAYS : OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, NULL);
return (p->handle != INVALID_HANDLE_VALUE) ? 0 : GetLastError();
}
WRes InFile_OpenW(CSzFile *p, const WCHAR *name) { return File_OpenW(p, name, 0); }
WRes OutFile_OpenW(CSzFile *p, const WCHAR *name) { return File_OpenW(p, name, 1); }
#endif
WRes File_Close(CSzFile *p)
{
#ifdef USE_WINDOWS_FILE
if (p->handle != INVALID_HANDLE_VALUE)
{
if (!CloseHandle(p->handle))
return GetLastError();
p->handle = INVALID_HANDLE_VALUE;
}
#else
if (p->file != NULL)
{
int res = fclose(p->file);
if (res != 0)
return res;
p->file = NULL;
}
#endif
return 0;
}
WRes File_Read(CSzFile *p, void *data, size_t *size)
{
size_t originalSize = *size;
if (originalSize == 0)
return 0;
#ifdef USE_WINDOWS_FILE
*size = 0;
do
{
DWORD curSize = (originalSize > kChunkSizeMax) ? kChunkSizeMax : (DWORD)originalSize;
DWORD processed = 0;
BOOL res = ReadFile(p->handle, data, curSize, &processed, NULL);
data = (void *)((Byte *)data + processed);
originalSize -= processed;
*size += processed;
if (!res)
return GetLastError();
if (processed == 0)
break;
}
while (originalSize > 0);
return 0;
#else
*size = fread(data, 1, originalSize, p->file);
if (*size == originalSize)
return 0;
return ferror(p->file);
#endif
}
WRes File_Write(CSzFile *p, const void *data, size_t *size)
{
size_t originalSize = *size;
if (originalSize == 0)
return 0;
#ifdef USE_WINDOWS_FILE
*size = 0;
do
{
DWORD curSize = (originalSize > kChunkSizeMax) ? kChunkSizeMax : (DWORD)originalSize;
DWORD processed = 0;
BOOL res = WriteFile(p->handle, data, curSize, &processed, NULL);
data = (void *)((Byte *)data + processed);
originalSize -= processed;
*size += processed;
if (!res)
return GetLastError();
if (processed == 0)
break;
}
while (originalSize > 0);
return 0;
#else
*size = fwrite(data, 1, originalSize, p->file);
if (*size == originalSize)
return 0;
return ferror(p->file);
#endif
}
WRes File_Seek(CSzFile *p, Int64 *pos, ESzSeek origin)
{
#ifdef USE_WINDOWS_FILE
LARGE_INTEGER value;
DWORD moveMethod;
value.LowPart = (DWORD)*pos;
value.HighPart = (LONG)((UInt64)*pos >> 16 >> 16); /* for case when UInt64 is 32-bit only */
switch (origin)
{
case SZ_SEEK_SET: moveMethod = FILE_BEGIN; break;
case SZ_SEEK_CUR: moveMethod = FILE_CURRENT; break;
case SZ_SEEK_END: moveMethod = FILE_END; break;
default: return ERROR_INVALID_PARAMETER;
}
value.LowPart = SetFilePointer(p->handle, value.LowPart, &value.HighPart, moveMethod);
if (value.LowPart == 0xFFFFFFFF)
{
WRes res = GetLastError();
if (res != NO_ERROR)
return res;
}
*pos = ((Int64)value.HighPart << 32) | value.LowPart;
return 0;
#else
int moveMethod;
int res;
switch (origin)
{
case SZ_SEEK_SET: moveMethod = SEEK_SET; break;
case SZ_SEEK_CUR: moveMethod = SEEK_CUR; break;
case SZ_SEEK_END: moveMethod = SEEK_END; break;
default: return 1;
}
res = fseek(p->file, (long)*pos, moveMethod);
*pos = ftell(p->file);
return res;
#endif
}
WRes File_GetLength(CSzFile *p, UInt64 *length)
{
#ifdef USE_WINDOWS_FILE
DWORD sizeHigh;
DWORD sizeLow = GetFileSize(p->handle, &sizeHigh);
if (sizeLow == 0xFFFFFFFF)
{
DWORD res = GetLastError();
if (res != NO_ERROR)
return res;
}
*length = (((UInt64)sizeHigh) << 32) + sizeLow;
return 0;
#else
long pos = ftell(p->file);
int res = fseek(p->file, 0, SEEK_END);
*length = ftell(p->file);
fseek(p->file, pos, SEEK_SET);
return res;
#endif
}
/* ---------- FileSeqInStream ---------- */
static SRes FileSeqInStream_Read(const ISeqInStream *pp, void *buf, size_t *size)
{
CFileSeqInStream *p = CONTAINER_FROM_VTBL(pp, CFileSeqInStream, vt);
return File_Read(&p->file, buf, size) == 0 ? SZ_OK : SZ_ERROR_READ;
}
void FileSeqInStream_CreateVTable(CFileSeqInStream *p)
{
p->vt.Read = FileSeqInStream_Read;
}
/* ---------- FileInStream ---------- */
static SRes FileInStream_Read(const ISeekInStream *pp, void *buf, size_t *size)
{
CFileInStream *p = CONTAINER_FROM_VTBL(pp, CFileInStream, vt);
return (File_Read(&p->file, buf, size) == 0) ? SZ_OK : SZ_ERROR_READ;
}
static SRes FileInStream_Seek(const ISeekInStream *pp, Int64 *pos, ESzSeek origin)
{
CFileInStream *p = CONTAINER_FROM_VTBL(pp, CFileInStream, vt);
return File_Seek(&p->file, pos, origin);
}
void FileInStream_CreateVTable(CFileInStream *p)
{
p->vt.Read = FileInStream_Read;
p->vt.Seek = FileInStream_Seek;
}
/* ---------- FileOutStream ---------- */
static size_t FileOutStream_Write(const ISeqOutStream *pp, const void *data, size_t size)
{
CFileOutStream *p = CONTAINER_FROM_VTBL(pp, CFileOutStream, vt);
File_Write(&p->file, data, &size);
return size;
}
void FileOutStream_CreateVTable(CFileOutStream *p)
{
p->vt.Write = FileOutStream_Write;
}
+83
View File
@@ -0,0 +1,83 @@
/* 7zFile.h -- File IO
2017-04-03 : Igor Pavlov : Public domain */
#ifndef __7Z_FILE_H
#define __7Z_FILE_H
#ifdef _WIN32
#define USE_WINDOWS_FILE
#endif
#ifdef USE_WINDOWS_FILE
#include <windows.h>
#else
#include <stdio.h>
#endif
#include "7zTypes.h"
EXTERN_C_BEGIN
/* ---------- File ---------- */
typedef struct
{
#ifdef USE_WINDOWS_FILE
HANDLE handle;
#else
FILE *file;
#endif
} CSzFile;
void File_Construct(CSzFile *p);
#if !defined(UNDER_CE) || !defined(USE_WINDOWS_FILE)
WRes InFile_Open(CSzFile *p, const char *name);
WRes OutFile_Open(CSzFile *p, const char *name);
#endif
#ifdef USE_WINDOWS_FILE
WRes InFile_OpenW(CSzFile *p, const WCHAR *name);
WRes OutFile_OpenW(CSzFile *p, const WCHAR *name);
#endif
WRes File_Close(CSzFile *p);
/* reads max(*size, remain file's size) bytes */
WRes File_Read(CSzFile *p, void *data, size_t *size);
/* writes *size bytes */
WRes File_Write(CSzFile *p, const void *data, size_t *size);
WRes File_Seek(CSzFile *p, Int64 *pos, ESzSeek origin);
WRes File_GetLength(CSzFile *p, UInt64 *length);
/* ---------- FileInStream ---------- */
typedef struct
{
ISeqInStream vt;
CSzFile file;
} CFileSeqInStream;
void FileSeqInStream_CreateVTable(CFileSeqInStream *p);
typedef struct
{
ISeekInStream vt;
CSzFile file;
} CFileInStream;
void FileInStream_CreateVTable(CFileInStream *p);
typedef struct
{
ISeqOutStream vt;
CSzFile file;
} CFileOutStream;
void FileOutStream_CreateVTable(CFileOutStream *p);
EXTERN_C_END
#endif
+176
View File
@@ -0,0 +1,176 @@
/* 7zStream.c -- 7z Stream functions
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <string.h>
#include "7zTypes.h"
SRes SeqInStream_Read2(const ISeqInStream *stream, void *buf, size_t size, SRes errorType)
{
while (size != 0)
{
size_t processed = size;
RINOK(ISeqInStream_Read(stream, buf, &processed));
if (processed == 0)
return errorType;
buf = (void *)((Byte *)buf + processed);
size -= processed;
}
return SZ_OK;
}
SRes SeqInStream_Read(const ISeqInStream *stream, void *buf, size_t size)
{
return SeqInStream_Read2(stream, buf, size, SZ_ERROR_INPUT_EOF);
}
SRes SeqInStream_ReadByte(const ISeqInStream *stream, Byte *buf)
{
size_t processed = 1;
RINOK(ISeqInStream_Read(stream, buf, &processed));
return (processed == 1) ? SZ_OK : SZ_ERROR_INPUT_EOF;
}
SRes LookInStream_SeekTo(const ILookInStream *stream, UInt64 offset)
{
Int64 t = offset;
return ILookInStream_Seek(stream, &t, SZ_SEEK_SET);
}
SRes LookInStream_LookRead(const ILookInStream *stream, void *buf, size_t *size)
{
const void *lookBuf;
if (*size == 0)
return SZ_OK;
RINOK(ILookInStream_Look(stream, &lookBuf, size));
memcpy(buf, lookBuf, *size);
return ILookInStream_Skip(stream, *size);
}
SRes LookInStream_Read2(const ILookInStream *stream, void *buf, size_t size, SRes errorType)
{
while (size != 0)
{
size_t processed = size;
RINOK(ILookInStream_Read(stream, buf, &processed));
if (processed == 0)
return errorType;
buf = (void *)((Byte *)buf + processed);
size -= processed;
}
return SZ_OK;
}
SRes LookInStream_Read(const ILookInStream *stream, void *buf, size_t size)
{
return LookInStream_Read2(stream, buf, size, SZ_ERROR_INPUT_EOF);
}
#define GET_LookToRead2 CLookToRead2 *p = CONTAINER_FROM_VTBL(pp, CLookToRead2, vt);
static SRes LookToRead2_Look_Lookahead(const ILookInStream *pp, const void **buf, size_t *size)
{
SRes res = SZ_OK;
GET_LookToRead2
size_t size2 = p->size - p->pos;
if (size2 == 0 && *size != 0)
{
p->pos = 0;
p->size = 0;
size2 = p->bufSize;
res = ISeekInStream_Read(p->realStream, p->buf, &size2);
p->size = size2;
}
if (*size > size2)
*size = size2;
*buf = p->buf + p->pos;
return res;
}
static SRes LookToRead2_Look_Exact(const ILookInStream *pp, const void **buf, size_t *size)
{
SRes res = SZ_OK;
GET_LookToRead2
size_t size2 = p->size - p->pos;
if (size2 == 0 && *size != 0)
{
p->pos = 0;
p->size = 0;
if (*size > p->bufSize)
*size = p->bufSize;
res = ISeekInStream_Read(p->realStream, p->buf, size);
size2 = p->size = *size;
}
if (*size > size2)
*size = size2;
*buf = p->buf + p->pos;
return res;
}
static SRes LookToRead2_Skip(const ILookInStream *pp, size_t offset)
{
GET_LookToRead2
p->pos += offset;
return SZ_OK;
}
static SRes LookToRead2_Read(const ILookInStream *pp, void *buf, size_t *size)
{
GET_LookToRead2
size_t rem = p->size - p->pos;
if (rem == 0)
return ISeekInStream_Read(p->realStream, buf, size);
if (rem > *size)
rem = *size;
memcpy(buf, p->buf + p->pos, rem);
p->pos += rem;
*size = rem;
return SZ_OK;
}
static SRes LookToRead2_Seek(const ILookInStream *pp, Int64 *pos, ESzSeek origin)
{
GET_LookToRead2
p->pos = p->size = 0;
return ISeekInStream_Seek(p->realStream, pos, origin);
}
void LookToRead2_CreateVTable(CLookToRead2 *p, int lookahead)
{
p->vt.Look = lookahead ?
LookToRead2_Look_Lookahead :
LookToRead2_Look_Exact;
p->vt.Skip = LookToRead2_Skip;
p->vt.Read = LookToRead2_Read;
p->vt.Seek = LookToRead2_Seek;
}
static SRes SecToLook_Read(const ISeqInStream *pp, void *buf, size_t *size)
{
CSecToLook *p = CONTAINER_FROM_VTBL(pp, CSecToLook, vt);
return LookInStream_LookRead(p->realStream, buf, size);
}
void SecToLook_CreateVTable(CSecToLook *p)
{
p->vt.Read = SecToLook_Read;
}
static SRes SecToRead_Read(const ISeqInStream *pp, void *buf, size_t *size)
{
CSecToRead *p = CONTAINER_FROM_VTBL(pp, CSecToRead, vt);
return ILookInStream_Read(p->realStream, buf, size);
}
void SecToRead_CreateVTable(CSecToRead *p)
{
p->vt.Read = SecToRead_Read;
}
+375
View File
@@ -0,0 +1,375 @@
/* 7zTypes.h -- Basic types
2018-08-04 : Igor Pavlov : Public domain */
#ifndef __7Z_TYPES_H
#define __7Z_TYPES_H
#ifdef _WIN32
/* #include <windows.h> */
#endif
#include <stddef.h>
#ifndef EXTERN_C_BEGIN
#ifdef __cplusplus
#define EXTERN_C_BEGIN extern "C" {
#define EXTERN_C_END }
#else
#define EXTERN_C_BEGIN
#define EXTERN_C_END
#endif
#endif
EXTERN_C_BEGIN
#define SZ_OK 0
#define SZ_ERROR_DATA 1
#define SZ_ERROR_MEM 2
#define SZ_ERROR_CRC 3
#define SZ_ERROR_UNSUPPORTED 4
#define SZ_ERROR_PARAM 5
#define SZ_ERROR_INPUT_EOF 6
#define SZ_ERROR_OUTPUT_EOF 7
#define SZ_ERROR_READ 8
#define SZ_ERROR_WRITE 9
#define SZ_ERROR_PROGRESS 10
#define SZ_ERROR_FAIL 11
#define SZ_ERROR_THREAD 12
#define SZ_ERROR_ARCHIVE 16
#define SZ_ERROR_NO_ARCHIVE 17
typedef int SRes;
#ifdef _WIN32
/* typedef DWORD WRes; */
typedef unsigned WRes;
#define MY_SRes_HRESULT_FROM_WRes(x) HRESULT_FROM_WIN32(x)
#else
typedef int WRes;
#define MY__FACILITY_WIN32 7
#define MY__FACILITY__WRes MY__FACILITY_WIN32
#define MY_SRes_HRESULT_FROM_WRes(x) ((HRESULT)(x) <= 0 ? ((HRESULT)(x)) : ((HRESULT) (((x) & 0x0000FFFF) | (MY__FACILITY__WRes << 16) | 0x80000000)))
#endif
#ifndef RINOK
#define RINOK(x) { int __result__ = (x); if (__result__ != 0) return __result__; }
#endif
typedef unsigned char Byte;
typedef short Int16;
typedef unsigned short UInt16;
#ifdef _LZMA_UINT32_IS_ULONG
typedef long Int32;
typedef unsigned long UInt32;
#else
typedef int Int32;
typedef unsigned int UInt32;
#endif
#ifdef _SZ_NO_INT_64
/* define _SZ_NO_INT_64, if your compiler doesn't support 64-bit integers.
NOTES: Some code will work incorrectly in that case! */
typedef long Int64;
typedef unsigned long UInt64;
#else
#if defined(_MSC_VER) || defined(__BORLANDC__)
typedef __int64 Int64;
typedef unsigned __int64 UInt64;
#define UINT64_CONST(n) n
#else
typedef long long int Int64;
typedef unsigned long long int UInt64;
#define UINT64_CONST(n) n ## ULL
#endif
#endif
#ifdef _LZMA_NO_SYSTEM_SIZE_T
typedef UInt32 SizeT;
#else
typedef size_t SizeT;
#endif
typedef int BoolInt;
/* typedef BoolInt Bool; */
#define True 1
#define False 0
#ifdef _WIN32
#define MY_STD_CALL __stdcall
#else
#define MY_STD_CALL
#endif
#ifdef _MSC_VER
#if _MSC_VER >= 1300
#define MY_NO_INLINE __declspec(noinline)
#else
#define MY_NO_INLINE
#endif
#define MY_FORCE_INLINE __forceinline
#define MY_CDECL __cdecl
#define MY_FAST_CALL __fastcall
#else
#define MY_NO_INLINE
#define MY_FORCE_INLINE
#define MY_CDECL
#define MY_FAST_CALL
/* inline keyword : for C++ / C99 */
/* GCC, clang: */
/*
#if defined (__GNUC__) && (__GNUC__ >= 4)
#define MY_FORCE_INLINE __attribute__((always_inline))
#define MY_NO_INLINE __attribute__((noinline))
#endif
*/
#endif
/* The following interfaces use first parameter as pointer to structure */
typedef struct IByteIn IByteIn;
struct IByteIn
{
Byte (*Read)(const IByteIn *p); /* reads one byte, returns 0 in case of EOF or error */
};
#define IByteIn_Read(p) (p)->Read(p)
typedef struct IByteOut IByteOut;
struct IByteOut
{
void (*Write)(const IByteOut *p, Byte b);
};
#define IByteOut_Write(p, b) (p)->Write(p, b)
typedef struct ISeqInStream ISeqInStream;
struct ISeqInStream
{
SRes (*Read)(const ISeqInStream *p, void *buf, size_t *size);
/* if (input(*size) != 0 && output(*size) == 0) means end_of_stream.
(output(*size) < input(*size)) is allowed */
};
#define ISeqInStream_Read(p, buf, size) (p)->Read(p, buf, size)
/* it can return SZ_ERROR_INPUT_EOF */
SRes SeqInStream_Read(const ISeqInStream *stream, void *buf, size_t size);
SRes SeqInStream_Read2(const ISeqInStream *stream, void *buf, size_t size, SRes errorType);
SRes SeqInStream_ReadByte(const ISeqInStream *stream, Byte *buf);
typedef struct ISeqOutStream ISeqOutStream;
struct ISeqOutStream
{
size_t (*Write)(const ISeqOutStream *p, const void *buf, size_t size);
/* Returns: result - the number of actually written bytes.
(result < size) means error */
};
#define ISeqOutStream_Write(p, buf, size) (p)->Write(p, buf, size)
typedef enum
{
SZ_SEEK_SET = 0,
SZ_SEEK_CUR = 1,
SZ_SEEK_END = 2
} ESzSeek;
typedef struct ISeekInStream ISeekInStream;
struct ISeekInStream
{
SRes (*Read)(const ISeekInStream *p, void *buf, size_t *size); /* same as ISeqInStream::Read */
SRes (*Seek)(const ISeekInStream *p, Int64 *pos, ESzSeek origin);
};
#define ISeekInStream_Read(p, buf, size) (p)->Read(p, buf, size)
#define ISeekInStream_Seek(p, pos, origin) (p)->Seek(p, pos, origin)
typedef struct ILookInStream ILookInStream;
struct ILookInStream
{
SRes (*Look)(const ILookInStream *p, const void **buf, size_t *size);
/* if (input(*size) != 0 && output(*size) == 0) means end_of_stream.
(output(*size) > input(*size)) is not allowed
(output(*size) < input(*size)) is allowed */
SRes (*Skip)(const ILookInStream *p, size_t offset);
/* offset must be <= output(*size) of Look */
SRes (*Read)(const ILookInStream *p, void *buf, size_t *size);
/* reads directly (without buffer). It's same as ISeqInStream::Read */
SRes (*Seek)(const ILookInStream *p, Int64 *pos, ESzSeek origin);
};
#define ILookInStream_Look(p, buf, size) (p)->Look(p, buf, size)
#define ILookInStream_Skip(p, offset) (p)->Skip(p, offset)
#define ILookInStream_Read(p, buf, size) (p)->Read(p, buf, size)
#define ILookInStream_Seek(p, pos, origin) (p)->Seek(p, pos, origin)
SRes LookInStream_LookRead(const ILookInStream *stream, void *buf, size_t *size);
SRes LookInStream_SeekTo(const ILookInStream *stream, UInt64 offset);
/* reads via ILookInStream::Read */
SRes LookInStream_Read2(const ILookInStream *stream, void *buf, size_t size, SRes errorType);
SRes LookInStream_Read(const ILookInStream *stream, void *buf, size_t size);
typedef struct
{
ILookInStream vt;
const ISeekInStream *realStream;
size_t pos;
size_t size; /* it's data size */
/* the following variables must be set outside */
Byte *buf;
size_t bufSize;
} CLookToRead2;
void LookToRead2_CreateVTable(CLookToRead2 *p, int lookahead);
#define LookToRead2_Init(p) { (p)->pos = (p)->size = 0; }
typedef struct
{
ISeqInStream vt;
const ILookInStream *realStream;
} CSecToLook;
void SecToLook_CreateVTable(CSecToLook *p);
typedef struct
{
ISeqInStream vt;
const ILookInStream *realStream;
} CSecToRead;
void SecToRead_CreateVTable(CSecToRead *p);
typedef struct ICompressProgress ICompressProgress;
struct ICompressProgress
{
SRes (*Progress)(const ICompressProgress *p, UInt64 inSize, UInt64 outSize);
/* Returns: result. (result != SZ_OK) means break.
Value (UInt64)(Int64)-1 for size means unknown value. */
};
#define ICompressProgress_Progress(p, inSize, outSize) (p)->Progress(p, inSize, outSize)
typedef struct ISzAlloc ISzAlloc;
typedef const ISzAlloc * ISzAllocPtr;
struct ISzAlloc
{
void *(*Alloc)(ISzAllocPtr p, size_t size);
void (*Free)(ISzAllocPtr p, void *address); /* address can be 0 */
};
#define ISzAlloc_Alloc(p, size) (p)->Alloc(p, size)
#define ISzAlloc_Free(p, a) (p)->Free(p, a)
/* deprecated */
#define IAlloc_Alloc(p, size) ISzAlloc_Alloc(p, size)
#define IAlloc_Free(p, a) ISzAlloc_Free(p, a)
#ifndef MY_offsetof
#ifdef offsetof
#define MY_offsetof(type, m) offsetof(type, m)
/*
#define MY_offsetof(type, m) FIELD_OFFSET(type, m)
*/
#else
#define MY_offsetof(type, m) ((size_t)&(((type *)0)->m))
#endif
#endif
#ifndef MY_container_of
/*
#define MY_container_of(ptr, type, m) container_of(ptr, type, m)
#define MY_container_of(ptr, type, m) CONTAINING_RECORD(ptr, type, m)
#define MY_container_of(ptr, type, m) ((type *)((char *)(ptr) - offsetof(type, m)))
#define MY_container_of(ptr, type, m) (&((type *)0)->m == (ptr), ((type *)(((char *)(ptr)) - MY_offsetof(type, m))))
*/
/*
GCC shows warning: "perhaps the 'offsetof' macro was used incorrectly"
GCC 3.4.4 : classes with constructor
GCC 4.8.1 : classes with non-public variable members"
*/
#define MY_container_of(ptr, type, m) ((type *)((char *)(1 ? (ptr) : &((type *)0)->m) - MY_offsetof(type, m)))
#endif
#define CONTAINER_FROM_VTBL_SIMPLE(ptr, type, m) ((type *)(ptr))
/*
#define CONTAINER_FROM_VTBL(ptr, type, m) CONTAINER_FROM_VTBL_SIMPLE(ptr, type, m)
*/
#define CONTAINER_FROM_VTBL(ptr, type, m) MY_container_of(ptr, type, m)
#define CONTAINER_FROM_VTBL_CLS(ptr, type, m) CONTAINER_FROM_VTBL_SIMPLE(ptr, type, m)
/*
#define CONTAINER_FROM_VTBL_CLS(ptr, type, m) CONTAINER_FROM_VTBL(ptr, type, m)
*/
#ifdef _WIN32
#define CHAR_PATH_SEPARATOR '\\'
#define WCHAR_PATH_SEPARATOR L'\\'
#define STRING_PATH_SEPARATOR "\\"
#define WSTRING_PATH_SEPARATOR L"\\"
#else
#define CHAR_PATH_SEPARATOR '/'
#define WCHAR_PATH_SEPARATOR L'/'
#define STRING_PATH_SEPARATOR "/"
#define WSTRING_PATH_SEPARATOR L"/"
#endif
EXTERN_C_END
#endif
+27
View File
@@ -0,0 +1,27 @@
#define MY_VER_MAJOR 19
#define MY_VER_MINOR 00
#define MY_VER_BUILD 0
#define MY_VERSION_NUMBERS "19.00"
#define MY_VERSION MY_VERSION_NUMBERS
#ifdef MY_CPU_NAME
#define MY_VERSION_CPU MY_VERSION " (" MY_CPU_NAME ")"
#else
#define MY_VERSION_CPU MY_VERSION
#endif
#define MY_DATE "2019-02-21"
#undef MY_COPYRIGHT
#undef MY_VERSION_COPYRIGHT_DATE
#define MY_AUTHOR_NAME "Igor Pavlov"
#define MY_COPYRIGHT_PD "Igor Pavlov : Public domain"
#define MY_COPYRIGHT_CR "Copyright (c) 1999-2018 Igor Pavlov"
#ifdef USE_COPYRIGHT_CR
#define MY_COPYRIGHT MY_COPYRIGHT_CR
#else
#define MY_COPYRIGHT MY_COPYRIGHT_PD
#endif
#define MY_COPYRIGHT_DATE MY_COPYRIGHT " : " MY_DATE
#define MY_VERSION_COPYRIGHT_DATE MY_VERSION_CPU " : " MY_COPYRIGHT " : " MY_DATE
+55
View File
@@ -0,0 +1,55 @@
#define MY_VS_FFI_FILEFLAGSMASK 0x0000003FL
#define MY_VOS_NT_WINDOWS32 0x00040004L
#define MY_VOS_CE_WINDOWS32 0x00050004L
#define MY_VFT_APP 0x00000001L
#define MY_VFT_DLL 0x00000002L
// #include <WinVer.h>
#ifndef MY_VERSION
#include "7zVersion.h"
#endif
#define MY_VER MY_VER_MAJOR,MY_VER_MINOR,MY_VER_BUILD,0
#ifdef DEBUG
#define DBG_FL VS_FF_DEBUG
#else
#define DBG_FL 0
#endif
#define MY_VERSION_INFO(fileType, descr, intName, origName) \
LANGUAGE 9, 1 \
1 VERSIONINFO \
FILEVERSION MY_VER \
PRODUCTVERSION MY_VER \
FILEFLAGSMASK MY_VS_FFI_FILEFLAGSMASK \
FILEFLAGS DBG_FL \
FILEOS MY_VOS_NT_WINDOWS32 \
FILETYPE fileType \
FILESUBTYPE 0x0L \
BEGIN \
BLOCK "StringFileInfo" \
BEGIN \
BLOCK "040904b0" \
BEGIN \
VALUE "CompanyName", "Igor Pavlov" \
VALUE "FileDescription", descr \
VALUE "FileVersion", MY_VERSION \
VALUE "InternalName", intName \
VALUE "LegalCopyright", MY_COPYRIGHT \
VALUE "OriginalFilename", origName \
VALUE "ProductName", "7-Zip" \
VALUE "ProductVersion", MY_VERSION \
END \
END \
BLOCK "VarFileInfo" \
BEGIN \
VALUE "Translation", 0x409, 1200 \
END \
END
#define MY_VERSION_INFO_APP(descr, intName) MY_VERSION_INFO(MY_VFT_APP, descr, intName, intName ".exe")
#define MY_VERSION_INFO_DLL(descr, intName) MY_VERSION_INFO(MY_VFT_DLL, descr, intName, intName ".dll")
+306
View File
@@ -0,0 +1,306 @@
/* Aes.c -- AES encryption / decryption
2017-01-24 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "Aes.h"
#include "CpuArch.h"
static UInt32 T[256 * 4];
static const Byte Sbox[256] = {
0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76,
0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0,
0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15,
0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75,
0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84,
0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf,
0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8,
0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2,
0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73,
0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb,
0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79,
0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08,
0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a,
0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e,
0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf,
0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16};
void MY_FAST_CALL AesCbc_Encode(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCbc_Decode(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCtr_Code(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCbc_Encode_Intel(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCbc_Decode_Intel(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCtr_Code_Intel(UInt32 *ivAes, Byte *data, size_t numBlocks);
AES_CODE_FUNC g_AesCbc_Encode;
AES_CODE_FUNC g_AesCbc_Decode;
AES_CODE_FUNC g_AesCtr_Code;
static UInt32 D[256 * 4];
static Byte InvS[256];
static const Byte Rcon[11] = { 0x00, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36 };
#define xtime(x) ((((x) << 1) ^ (((x) & 0x80) != 0 ? 0x1B : 0)) & 0xFF)
#define Ui32(a0, a1, a2, a3) ((UInt32)(a0) | ((UInt32)(a1) << 8) | ((UInt32)(a2) << 16) | ((UInt32)(a3) << 24))
#define gb0(x) ( (x) & 0xFF)
#define gb1(x) (((x) >> ( 8)) & 0xFF)
#define gb2(x) (((x) >> (16)) & 0xFF)
#define gb3(x) (((x) >> (24)))
#define gb(n, x) gb ## n(x)
#define TT(x) (T + (x << 8))
#define DD(x) (D + (x << 8))
void AesGenTables(void)
{
unsigned i;
for (i = 0; i < 256; i++)
InvS[Sbox[i]] = (Byte)i;
for (i = 0; i < 256; i++)
{
{
UInt32 a1 = Sbox[i];
UInt32 a2 = xtime(a1);
UInt32 a3 = a2 ^ a1;
TT(0)[i] = Ui32(a2, a1, a1, a3);
TT(1)[i] = Ui32(a3, a2, a1, a1);
TT(2)[i] = Ui32(a1, a3, a2, a1);
TT(3)[i] = Ui32(a1, a1, a3, a2);
}
{
UInt32 a1 = InvS[i];
UInt32 a2 = xtime(a1);
UInt32 a4 = xtime(a2);
UInt32 a8 = xtime(a4);
UInt32 a9 = a8 ^ a1;
UInt32 aB = a8 ^ a2 ^ a1;
UInt32 aD = a8 ^ a4 ^ a1;
UInt32 aE = a8 ^ a4 ^ a2;
DD(0)[i] = Ui32(aE, a9, aD, aB);
DD(1)[i] = Ui32(aB, aE, a9, aD);
DD(2)[i] = Ui32(aD, aB, aE, a9);
DD(3)[i] = Ui32(a9, aD, aB, aE);
}
}
g_AesCbc_Encode = AesCbc_Encode;
g_AesCbc_Decode = AesCbc_Decode;
g_AesCtr_Code = AesCtr_Code;
#ifdef MY_CPU_X86_OR_AMD64
if (CPU_Is_Aes_Supported())
{
g_AesCbc_Encode = AesCbc_Encode_Intel;
g_AesCbc_Decode = AesCbc_Decode_Intel;
g_AesCtr_Code = AesCtr_Code_Intel;
}
#endif
}
#define HT(i, x, s) TT(x)[gb(x, s[(i + x) & 3])]
#define HT4(m, i, s, p) m[i] = \
HT(i, 0, s) ^ \
HT(i, 1, s) ^ \
HT(i, 2, s) ^ \
HT(i, 3, s) ^ w[p + i]
#define HT16(m, s, p) \
HT4(m, 0, s, p); \
HT4(m, 1, s, p); \
HT4(m, 2, s, p); \
HT4(m, 3, s, p); \
#define FT(i, x) Sbox[gb(x, m[(i + x) & 3])]
#define FT4(i) dest[i] = Ui32(FT(i, 0), FT(i, 1), FT(i, 2), FT(i, 3)) ^ w[i];
#define HD(i, x, s) DD(x)[gb(x, s[(i - x) & 3])]
#define HD4(m, i, s, p) m[i] = \
HD(i, 0, s) ^ \
HD(i, 1, s) ^ \
HD(i, 2, s) ^ \
HD(i, 3, s) ^ w[p + i];
#define HD16(m, s, p) \
HD4(m, 0, s, p); \
HD4(m, 1, s, p); \
HD4(m, 2, s, p); \
HD4(m, 3, s, p); \
#define FD(i, x) InvS[gb(x, m[(i - x) & 3])]
#define FD4(i) dest[i] = Ui32(FD(i, 0), FD(i, 1), FD(i, 2), FD(i, 3)) ^ w[i];
void MY_FAST_CALL Aes_SetKey_Enc(UInt32 *w, const Byte *key, unsigned keySize)
{
unsigned i, wSize;
wSize = keySize + 28;
keySize /= 4;
w[0] = ((UInt32)keySize / 2) + 3;
w += 4;
for (i = 0; i < keySize; i++, key += 4)
w[i] = GetUi32(key);
for (; i < wSize; i++)
{
UInt32 t = w[(size_t)i - 1];
unsigned rem = i % keySize;
if (rem == 0)
t = Ui32(Sbox[gb1(t)] ^ Rcon[i / keySize], Sbox[gb2(t)], Sbox[gb3(t)], Sbox[gb0(t)]);
else if (keySize > 6 && rem == 4)
t = Ui32(Sbox[gb0(t)], Sbox[gb1(t)], Sbox[gb2(t)], Sbox[gb3(t)]);
w[i] = w[i - keySize] ^ t;
}
}
void MY_FAST_CALL Aes_SetKey_Dec(UInt32 *w, const Byte *key, unsigned keySize)
{
unsigned i, num;
Aes_SetKey_Enc(w, key, keySize);
num = keySize + 20;
w += 8;
for (i = 0; i < num; i++)
{
UInt32 r = w[i];
w[i] =
DD(0)[Sbox[gb0(r)]] ^
DD(1)[Sbox[gb1(r)]] ^
DD(2)[Sbox[gb2(r)]] ^
DD(3)[Sbox[gb3(r)]];
}
}
/* Aes_Encode and Aes_Decode functions work with little-endian words.
src and dest are pointers to 4 UInt32 words.
src and dest can point to same block */
static void Aes_Encode(const UInt32 *w, UInt32 *dest, const UInt32 *src)
{
UInt32 s[4];
UInt32 m[4];
UInt32 numRounds2 = w[0];
w += 4;
s[0] = src[0] ^ w[0];
s[1] = src[1] ^ w[1];
s[2] = src[2] ^ w[2];
s[3] = src[3] ^ w[3];
w += 4;
for (;;)
{
HT16(m, s, 0);
if (--numRounds2 == 0)
break;
HT16(s, m, 4);
w += 8;
}
w += 4;
FT4(0); FT4(1); FT4(2); FT4(3);
}
static void Aes_Decode(const UInt32 *w, UInt32 *dest, const UInt32 *src)
{
UInt32 s[4];
UInt32 m[4];
UInt32 numRounds2 = w[0];
w += 4 + numRounds2 * 8;
s[0] = src[0] ^ w[0];
s[1] = src[1] ^ w[1];
s[2] = src[2] ^ w[2];
s[3] = src[3] ^ w[3];
for (;;)
{
w -= 8;
HD16(m, s, 4);
if (--numRounds2 == 0)
break;
HD16(s, m, 0);
}
FD4(0); FD4(1); FD4(2); FD4(3);
}
void AesCbc_Init(UInt32 *p, const Byte *iv)
{
unsigned i;
for (i = 0; i < 4; i++)
p[i] = GetUi32(iv + i * 4);
}
void MY_FAST_CALL AesCbc_Encode(UInt32 *p, Byte *data, size_t numBlocks)
{
for (; numBlocks != 0; numBlocks--, data += AES_BLOCK_SIZE)
{
p[0] ^= GetUi32(data);
p[1] ^= GetUi32(data + 4);
p[2] ^= GetUi32(data + 8);
p[3] ^= GetUi32(data + 12);
Aes_Encode(p + 4, p, p);
SetUi32(data, p[0]);
SetUi32(data + 4, p[1]);
SetUi32(data + 8, p[2]);
SetUi32(data + 12, p[3]);
}
}
void MY_FAST_CALL AesCbc_Decode(UInt32 *p, Byte *data, size_t numBlocks)
{
UInt32 in[4], out[4];
for (; numBlocks != 0; numBlocks--, data += AES_BLOCK_SIZE)
{
in[0] = GetUi32(data);
in[1] = GetUi32(data + 4);
in[2] = GetUi32(data + 8);
in[3] = GetUi32(data + 12);
Aes_Decode(p + 4, out, in);
SetUi32(data, p[0] ^ out[0]);
SetUi32(data + 4, p[1] ^ out[1]);
SetUi32(data + 8, p[2] ^ out[2]);
SetUi32(data + 12, p[3] ^ out[3]);
p[0] = in[0];
p[1] = in[1];
p[2] = in[2];
p[3] = in[3];
}
}
void MY_FAST_CALL AesCtr_Code(UInt32 *p, Byte *data, size_t numBlocks)
{
for (; numBlocks != 0; numBlocks--)
{
UInt32 temp[4];
unsigned i;
if (++p[0] == 0)
p[1]++;
Aes_Encode(p + 4, temp, p);
for (i = 0; i < 4; i++, data += 4)
{
UInt32 t = temp[i];
#ifdef MY_CPU_LE_UNALIGN
*((UInt32 *)data) ^= t;
#else
data[0] ^= (t & 0xFF);
data[1] ^= ((t >> 8) & 0xFF);
data[2] ^= ((t >> 16) & 0xFF);
data[3] ^= ((t >> 24));
#endif
}
}
}
+38
View File
@@ -0,0 +1,38 @@
/* Aes.h -- AES encryption / decryption
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __AES_H
#define __AES_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define AES_BLOCK_SIZE 16
/* Call AesGenTables one time before other AES functions */
void AesGenTables(void);
/* UInt32 pointers must be 16-byte aligned */
/* 16-byte (4 * 32-bit words) blocks: 1 (IV) + 1 (keyMode) + 15 (AES-256 roundKeys) */
#define AES_NUM_IVMRK_WORDS ((1 + 1 + 15) * 4)
/* aes - 16-byte aligned pointer to keyMode+roundKeys sequence */
/* keySize = 16 or 24 or 32 (bytes) */
typedef void (MY_FAST_CALL *AES_SET_KEY_FUNC)(UInt32 *aes, const Byte *key, unsigned keySize);
void MY_FAST_CALL Aes_SetKey_Enc(UInt32 *aes, const Byte *key, unsigned keySize);
void MY_FAST_CALL Aes_SetKey_Dec(UInt32 *aes, const Byte *key, unsigned keySize);
/* ivAes - 16-byte aligned pointer to iv+keyMode+roundKeys sequence: UInt32[AES_NUM_IVMRK_WORDS] */
void AesCbc_Init(UInt32 *ivAes, const Byte *iv); /* iv size is AES_BLOCK_SIZE */
/* data - 16-byte aligned pointer to data */
/* numBlocks - the number of 16-byte blocks in data array */
typedef void (MY_FAST_CALL *AES_CODE_FUNC)(UInt32 *ivAes, Byte *data, size_t numBlocks);
extern AES_CODE_FUNC g_AesCbc_Encode;
extern AES_CODE_FUNC g_AesCbc_Decode;
extern AES_CODE_FUNC g_AesCtr_Code;
EXTERN_C_END
#endif
+184
View File
@@ -0,0 +1,184 @@
/* AesOpt.c -- Intel's AES
2017-06-08 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "CpuArch.h"
#ifdef MY_CPU_X86_OR_AMD64
#if (_MSC_VER > 1500) || (_MSC_FULL_VER >= 150030729)
#define USE_INTEL_AES
#endif
#endif
#ifdef USE_INTEL_AES
#include <wmmintrin.h>
void MY_FAST_CALL AesCbc_Encode_Intel(__m128i *p, __m128i *data, size_t numBlocks)
{
__m128i m = *p;
for (; numBlocks != 0; numBlocks--, data++)
{
UInt32 numRounds2 = *(const UInt32 *)(p + 1) - 1;
const __m128i *w = p + 3;
m = _mm_xor_si128(m, *data);
m = _mm_xor_si128(m, p[2]);
do
{
m = _mm_aesenc_si128(m, w[0]);
m = _mm_aesenc_si128(m, w[1]);
w += 2;
}
while (--numRounds2 != 0);
m = _mm_aesenc_si128(m, w[0]);
m = _mm_aesenclast_si128(m, w[1]);
*data = m;
}
*p = m;
}
#define NUM_WAYS 3
#define AES_OP_W(op, n) { \
const __m128i t = w[n]; \
m0 = op(m0, t); \
m1 = op(m1, t); \
m2 = op(m2, t); \
}
#define AES_DEC(n) AES_OP_W(_mm_aesdec_si128, n)
#define AES_DEC_LAST(n) AES_OP_W(_mm_aesdeclast_si128, n)
#define AES_ENC(n) AES_OP_W(_mm_aesenc_si128, n)
#define AES_ENC_LAST(n) AES_OP_W(_mm_aesenclast_si128, n)
void MY_FAST_CALL AesCbc_Decode_Intel(__m128i *p, __m128i *data, size_t numBlocks)
{
__m128i iv = *p;
for (; numBlocks >= NUM_WAYS; numBlocks -= NUM_WAYS, data += NUM_WAYS)
{
UInt32 numRounds2 = *(const UInt32 *)(p + 1);
const __m128i *w = p + numRounds2 * 2;
__m128i m0, m1, m2;
{
const __m128i t = w[2];
m0 = _mm_xor_si128(t, data[0]);
m1 = _mm_xor_si128(t, data[1]);
m2 = _mm_xor_si128(t, data[2]);
}
numRounds2--;
do
{
AES_DEC(1)
AES_DEC(0)
w -= 2;
}
while (--numRounds2 != 0);
AES_DEC(1)
AES_DEC_LAST(0)
{
__m128i t;
t = _mm_xor_si128(m0, iv); iv = data[0]; data[0] = t;
t = _mm_xor_si128(m1, iv); iv = data[1]; data[1] = t;
t = _mm_xor_si128(m2, iv); iv = data[2]; data[2] = t;
}
}
for (; numBlocks != 0; numBlocks--, data++)
{
UInt32 numRounds2 = *(const UInt32 *)(p + 1);
const __m128i *w = p + numRounds2 * 2;
__m128i m = _mm_xor_si128(w[2], *data);
numRounds2--;
do
{
m = _mm_aesdec_si128(m, w[1]);
m = _mm_aesdec_si128(m, w[0]);
w -= 2;
}
while (--numRounds2 != 0);
m = _mm_aesdec_si128(m, w[1]);
m = _mm_aesdeclast_si128(m, w[0]);
m = _mm_xor_si128(m, iv);
iv = *data;
*data = m;
}
*p = iv;
}
void MY_FAST_CALL AesCtr_Code_Intel(__m128i *p, __m128i *data, size_t numBlocks)
{
__m128i ctr = *p;
__m128i one;
one.m128i_u64[0] = 1;
one.m128i_u64[1] = 0;
for (; numBlocks >= NUM_WAYS; numBlocks -= NUM_WAYS, data += NUM_WAYS)
{
UInt32 numRounds2 = *(const UInt32 *)(p + 1) - 1;
const __m128i *w = p;
__m128i m0, m1, m2;
{
const __m128i t = w[2];
ctr = _mm_add_epi64(ctr, one); m0 = _mm_xor_si128(ctr, t);
ctr = _mm_add_epi64(ctr, one); m1 = _mm_xor_si128(ctr, t);
ctr = _mm_add_epi64(ctr, one); m2 = _mm_xor_si128(ctr, t);
}
w += 3;
do
{
AES_ENC(0)
AES_ENC(1)
w += 2;
}
while (--numRounds2 != 0);
AES_ENC(0)
AES_ENC_LAST(1)
data[0] = _mm_xor_si128(data[0], m0);
data[1] = _mm_xor_si128(data[1], m1);
data[2] = _mm_xor_si128(data[2], m2);
}
for (; numBlocks != 0; numBlocks--, data++)
{
UInt32 numRounds2 = *(const UInt32 *)(p + 1) - 1;
const __m128i *w = p;
__m128i m;
ctr = _mm_add_epi64(ctr, one);
m = _mm_xor_si128(ctr, p[2]);
w += 3;
do
{
m = _mm_aesenc_si128(m, w[0]);
m = _mm_aesenc_si128(m, w[1]);
w += 2;
}
while (--numRounds2 != 0);
m = _mm_aesenc_si128(m, w[0]);
m = _mm_aesenclast_si128(m, w[1]);
*data = _mm_xor_si128(*data, m);
}
*p = ctr;
}
#else
void MY_FAST_CALL AesCbc_Encode(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCbc_Decode(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCtr_Code(UInt32 *ivAes, Byte *data, size_t numBlocks);
void MY_FAST_CALL AesCbc_Encode_Intel(UInt32 *p, Byte *data, size_t numBlocks)
{
AesCbc_Encode(p, data, numBlocks);
}
void MY_FAST_CALL AesCbc_Decode_Intel(UInt32 *p, Byte *data, size_t numBlocks)
{
AesCbc_Decode(p, data, numBlocks);
}
void MY_FAST_CALL AesCtr_Code_Intel(UInt32 *p, Byte *data, size_t numBlocks)
{
AesCtr_Code(p, data, numBlocks);
}
#endif
+455
View File
@@ -0,0 +1,455 @@
/* Alloc.c -- Memory allocation functions
2018-04-27 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <stdio.h>
#ifdef _WIN32
#include <windows.h>
#endif
#include <stdlib.h>
#include "Alloc.h"
/* #define _SZ_ALLOC_DEBUG */
/* use _SZ_ALLOC_DEBUG to debug alloc/free operations */
#ifdef _SZ_ALLOC_DEBUG
#include <stdio.h>
int g_allocCount = 0;
int g_allocCountMid = 0;
int g_allocCountBig = 0;
#define CONVERT_INT_TO_STR(charType, tempSize) \
unsigned char temp[tempSize]; unsigned i = 0; \
while (val >= 10) { temp[i++] = (unsigned char)('0' + (unsigned)(val % 10)); val /= 10; } \
*s++ = (charType)('0' + (unsigned)val); \
while (i != 0) { i--; *s++ = temp[i]; } \
*s = 0;
static void ConvertUInt64ToString(UInt64 val, char *s)
{
CONVERT_INT_TO_STR(char, 24);
}
#define GET_HEX_CHAR(t) ((char)(((t < 10) ? ('0' + t) : ('A' + (t - 10)))))
static void ConvertUInt64ToHex(UInt64 val, char *s)
{
UInt64 v = val;
unsigned i;
for (i = 1;; i++)
{
v >>= 4;
if (v == 0)
break;
}
s[i] = 0;
do
{
unsigned t = (unsigned)(val & 0xF);
val >>= 4;
s[--i] = GET_HEX_CHAR(t);
}
while (i);
}
#define DEBUG_OUT_STREAM stderr
static void Print(const char *s)
{
fputs(s, DEBUG_OUT_STREAM);
}
static void PrintAligned(const char *s, size_t align)
{
size_t len = strlen(s);
for(;;)
{
fputc(' ', DEBUG_OUT_STREAM);
if (len >= align)
break;
++len;
}
Print(s);
}
static void PrintLn()
{
Print("\n");
}
static void PrintHex(UInt64 v, size_t align)
{
char s[32];
ConvertUInt64ToHex(v, s);
PrintAligned(s, align);
}
static void PrintDec(UInt64 v, size_t align)
{
char s[32];
ConvertUInt64ToString(v, s);
PrintAligned(s, align);
}
static void PrintAddr(void *p)
{
PrintHex((UInt64)(size_t)(ptrdiff_t)p, 12);
}
#define PRINT_ALLOC(name, cnt, size, ptr) \
Print(name " "); \
PrintDec(cnt++, 10); \
PrintHex(size, 10); \
PrintAddr(ptr); \
PrintLn();
#define PRINT_FREE(name, cnt, ptr) if (ptr) { \
Print(name " "); \
PrintDec(--cnt, 10); \
PrintAddr(ptr); \
PrintLn(); }
#else
#define PRINT_ALLOC(name, cnt, size, ptr)
#define PRINT_FREE(name, cnt, ptr)
#define Print(s)
#define PrintLn()
#define PrintHex(v, align)
#define PrintDec(v, align)
#define PrintAddr(p)
#endif
void *MyAlloc(size_t size)
{
if (size == 0)
return NULL;
#ifdef _SZ_ALLOC_DEBUG
{
void *p = malloc(size);
PRINT_ALLOC("Alloc ", g_allocCount, size, p);
return p;
}
#else
return malloc(size);
#endif
}
void MyFree(void *address)
{
PRINT_FREE("Free ", g_allocCount, address);
free(address);
}
#ifdef _WIN32
void *MidAlloc(size_t size)
{
if (size == 0)
return NULL;
PRINT_ALLOC("Alloc-Mid", g_allocCountMid, size, NULL);
return VirtualAlloc(NULL, size, MEM_COMMIT, PAGE_READWRITE);
}
void MidFree(void *address)
{
PRINT_FREE("Free-Mid", g_allocCountMid, address);
if (!address)
return;
VirtualFree(address, 0, MEM_RELEASE);
}
#ifndef MEM_LARGE_PAGES
#undef _7ZIP_LARGE_PAGES
#endif
#ifdef _7ZIP_LARGE_PAGES
SIZE_T g_LargePageSize = 0;
typedef SIZE_T (WINAPI *GetLargePageMinimumP)();
#endif
void SetLargePageSize()
{
#ifdef _7ZIP_LARGE_PAGES
SIZE_T size;
GetLargePageMinimumP largePageMinimum = (GetLargePageMinimumP)
GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "GetLargePageMinimum");
if (!largePageMinimum)
return;
size = largePageMinimum();
if (size == 0 || (size & (size - 1)) != 0)
return;
g_LargePageSize = size;
#endif
}
void *BigAlloc(size_t size)
{
if (size == 0)
return NULL;
PRINT_ALLOC("Alloc-Big", g_allocCountBig, size, NULL);
#ifdef _7ZIP_LARGE_PAGES
{
SIZE_T ps = g_LargePageSize;
if (ps != 0 && ps <= (1 << 30) && size > (ps / 2))
{
size_t size2;
ps--;
size2 = (size + ps) & ~ps;
if (size2 >= size)
{
void *res = VirtualAlloc(NULL, size2, MEM_COMMIT | MEM_LARGE_PAGES, PAGE_READWRITE);
if (res)
return res;
}
}
}
#endif
return VirtualAlloc(NULL, size, MEM_COMMIT, PAGE_READWRITE);
}
void BigFree(void *address)
{
PRINT_FREE("Free-Big", g_allocCountBig, address);
if (!address)
return;
VirtualFree(address, 0, MEM_RELEASE);
}
#endif
static void *SzAlloc(ISzAllocPtr p, size_t size) { UNUSED_VAR(p); return MyAlloc(size); }
static void SzFree(ISzAllocPtr p, void *address) { UNUSED_VAR(p); MyFree(address); }
const ISzAlloc g_Alloc = { SzAlloc, SzFree };
static void *SzMidAlloc(ISzAllocPtr p, size_t size) { UNUSED_VAR(p); return MidAlloc(size); }
static void SzMidFree(ISzAllocPtr p, void *address) { UNUSED_VAR(p); MidFree(address); }
const ISzAlloc g_MidAlloc = { SzMidAlloc, SzMidFree };
static void *SzBigAlloc(ISzAllocPtr p, size_t size) { UNUSED_VAR(p); return BigAlloc(size); }
static void SzBigFree(ISzAllocPtr p, void *address) { UNUSED_VAR(p); BigFree(address); }
const ISzAlloc g_BigAlloc = { SzBigAlloc, SzBigFree };
/*
uintptr_t : <stdint.h> C99 (optional)
: unsupported in VS6
*/
#ifdef _WIN32
typedef UINT_PTR UIntPtr;
#else
/*
typedef uintptr_t UIntPtr;
*/
typedef ptrdiff_t UIntPtr;
#endif
#define ADJUST_ALLOC_SIZE 0
/*
#define ADJUST_ALLOC_SIZE (sizeof(void *) - 1)
*/
/*
Use (ADJUST_ALLOC_SIZE = (sizeof(void *) - 1)), if
MyAlloc() can return address that is NOT multiple of sizeof(void *).
*/
/*
#define MY_ALIGN_PTR_DOWN(p, align) ((void *)((char *)(p) - ((size_t)(UIntPtr)(p) & ((align) - 1))))
*/
#define MY_ALIGN_PTR_DOWN(p, align) ((void *)((((UIntPtr)(p)) & ~((UIntPtr)(align) - 1))))
#define MY_ALIGN_PTR_UP_PLUS(p, align) MY_ALIGN_PTR_DOWN(((char *)(p) + (align) + ADJUST_ALLOC_SIZE), align)
#if (_POSIX_C_SOURCE >= 200112L) && !defined(_WIN32)
#define USE_posix_memalign
#endif
/*
This posix_memalign() is for test purposes only.
We also need special Free() function instead of free(),
if this posix_memalign() is used.
*/
/*
static int posix_memalign(void **ptr, size_t align, size_t size)
{
size_t newSize = size + align;
void *p;
void *pAligned;
*ptr = NULL;
if (newSize < size)
return 12; // ENOMEM
p = MyAlloc(newSize);
if (!p)
return 12; // ENOMEM
pAligned = MY_ALIGN_PTR_UP_PLUS(p, align);
((void **)pAligned)[-1] = p;
*ptr = pAligned;
return 0;
}
*/
/*
ALLOC_ALIGN_SIZE >= sizeof(void *)
ALLOC_ALIGN_SIZE >= cache_line_size
*/
#define ALLOC_ALIGN_SIZE ((size_t)1 << 7)
static void *SzAlignedAlloc(ISzAllocPtr pp, size_t size)
{
#ifndef USE_posix_memalign
void *p;
void *pAligned;
size_t newSize;
UNUSED_VAR(pp);
/* also we can allocate additional dummy ALLOC_ALIGN_SIZE bytes after aligned
block to prevent cache line sharing with another allocated blocks */
newSize = size + ALLOC_ALIGN_SIZE * 1 + ADJUST_ALLOC_SIZE;
if (newSize < size)
return NULL;
p = MyAlloc(newSize);
if (!p)
return NULL;
pAligned = MY_ALIGN_PTR_UP_PLUS(p, ALLOC_ALIGN_SIZE);
Print(" size="); PrintHex(size, 8);
Print(" a_size="); PrintHex(newSize, 8);
Print(" ptr="); PrintAddr(p);
Print(" a_ptr="); PrintAddr(pAligned);
PrintLn();
((void **)pAligned)[-1] = p;
return pAligned;
#else
void *p;
UNUSED_VAR(pp);
if (posix_memalign(&p, ALLOC_ALIGN_SIZE, size))
return NULL;
Print(" posix_memalign="); PrintAddr(p);
PrintLn();
return p;
#endif
}
static void SzAlignedFree(ISzAllocPtr pp, void *address)
{
UNUSED_VAR(pp);
#ifndef USE_posix_memalign
if (address)
MyFree(((void **)address)[-1]);
#else
free(address);
#endif
}
const ISzAlloc g_AlignedAlloc = { SzAlignedAlloc, SzAlignedFree };
#define MY_ALIGN_PTR_DOWN_1(p) MY_ALIGN_PTR_DOWN(p, sizeof(void *))
/* we align ptr to support cases where CAlignOffsetAlloc::offset is not multiply of sizeof(void *) */
#define REAL_BLOCK_PTR_VAR(p) ((void **)MY_ALIGN_PTR_DOWN_1(p))[-1]
/*
#define REAL_BLOCK_PTR_VAR(p) ((void **)(p))[-1]
*/
static void *AlignOffsetAlloc_Alloc(ISzAllocPtr pp, size_t size)
{
CAlignOffsetAlloc *p = CONTAINER_FROM_VTBL(pp, CAlignOffsetAlloc, vt);
void *adr;
void *pAligned;
size_t newSize;
size_t extra;
size_t alignSize = (size_t)1 << p->numAlignBits;
if (alignSize < sizeof(void *))
alignSize = sizeof(void *);
if (p->offset >= alignSize)
return NULL;
/* also we can allocate additional dummy ALLOC_ALIGN_SIZE bytes after aligned
block to prevent cache line sharing with another allocated blocks */
extra = p->offset & (sizeof(void *) - 1);
newSize = size + alignSize + extra + ADJUST_ALLOC_SIZE;
if (newSize < size)
return NULL;
adr = ISzAlloc_Alloc(p->baseAlloc, newSize);
if (!adr)
return NULL;
pAligned = (char *)MY_ALIGN_PTR_DOWN((char *)adr +
alignSize - p->offset + extra + ADJUST_ALLOC_SIZE, alignSize) + p->offset;
PrintLn();
Print("- Aligned: ");
Print(" size="); PrintHex(size, 8);
Print(" a_size="); PrintHex(newSize, 8);
Print(" ptr="); PrintAddr(adr);
Print(" a_ptr="); PrintAddr(pAligned);
PrintLn();
REAL_BLOCK_PTR_VAR(pAligned) = adr;
return pAligned;
}
static void AlignOffsetAlloc_Free(ISzAllocPtr pp, void *address)
{
if (address)
{
CAlignOffsetAlloc *p = CONTAINER_FROM_VTBL(pp, CAlignOffsetAlloc, vt);
PrintLn();
Print("- Aligned Free: ");
PrintLn();
ISzAlloc_Free(p->baseAlloc, REAL_BLOCK_PTR_VAR(address));
}
}
void AlignOffsetAlloc_CreateVTable(CAlignOffsetAlloc *p)
{
p->vt.Alloc = AlignOffsetAlloc_Alloc;
p->vt.Free = AlignOffsetAlloc_Free;
}
+51
View File
@@ -0,0 +1,51 @@
/* Alloc.h -- Memory allocation functions
2018-02-19 : Igor Pavlov : Public domain */
#ifndef __COMMON_ALLOC_H
#define __COMMON_ALLOC_H
#include "7zTypes.h"
EXTERN_C_BEGIN
void *MyAlloc(size_t size);
void MyFree(void *address);
#ifdef _WIN32
void SetLargePageSize();
void *MidAlloc(size_t size);
void MidFree(void *address);
void *BigAlloc(size_t size);
void BigFree(void *address);
#else
#define MidAlloc(size) MyAlloc(size)
#define MidFree(address) MyFree(address)
#define BigAlloc(size) MyAlloc(size)
#define BigFree(address) MyFree(address)
#endif
extern const ISzAlloc g_Alloc;
extern const ISzAlloc g_BigAlloc;
extern const ISzAlloc g_MidAlloc;
extern const ISzAlloc g_AlignedAlloc;
typedef struct
{
ISzAlloc vt;
ISzAllocPtr baseAlloc;
unsigned numAlignBits; /* ((1 << numAlignBits) >= sizeof(void *)) */
size_t offset; /* (offset == (k * sizeof(void *)) && offset < (1 << numAlignBits) */
} CAlignOffsetAlloc;
void AlignOffsetAlloc_CreateVTable(CAlignOffsetAlloc *p);
EXTERN_C_END
#endif
+257
View File
@@ -0,0 +1,257 @@
/* Bcj2.c -- BCJ2 Decoder (Converter for x86 code)
2018-04-28 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "Bcj2.h"
#include "CpuArch.h"
#define CProb UInt16
#define kTopValue ((UInt32)1 << 24)
#define kNumModelBits 11
#define kBitModelTotal (1 << kNumModelBits)
#define kNumMoveBits 5
#define _IF_BIT_0 ttt = *prob; bound = (p->range >> kNumModelBits) * ttt; if (p->code < bound)
#define _UPDATE_0 p->range = bound; *prob = (CProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
#define _UPDATE_1 p->range -= bound; p->code -= bound; *prob = (CProb)(ttt - (ttt >> kNumMoveBits));
void Bcj2Dec_Init(CBcj2Dec *p)
{
unsigned i;
p->state = BCJ2_DEC_STATE_OK;
p->ip = 0;
p->temp[3] = 0;
p->range = 0;
p->code = 0;
for (i = 0; i < sizeof(p->probs) / sizeof(p->probs[0]); i++)
p->probs[i] = kBitModelTotal >> 1;
}
SRes Bcj2Dec_Decode(CBcj2Dec *p)
{
if (p->range <= 5)
{
p->state = BCJ2_DEC_STATE_OK;
for (; p->range != 5; p->range++)
{
if (p->range == 1 && p->code != 0)
return SZ_ERROR_DATA;
if (p->bufs[BCJ2_STREAM_RC] == p->lims[BCJ2_STREAM_RC])
{
p->state = BCJ2_STREAM_RC;
return SZ_OK;
}
p->code = (p->code << 8) | *(p->bufs[BCJ2_STREAM_RC])++;
}
if (p->code == 0xFFFFFFFF)
return SZ_ERROR_DATA;
p->range = 0xFFFFFFFF;
}
else if (p->state >= BCJ2_DEC_STATE_ORIG_0)
{
while (p->state <= BCJ2_DEC_STATE_ORIG_3)
{
Byte *dest = p->dest;
if (dest == p->destLim)
return SZ_OK;
*dest = p->temp[(size_t)p->state - BCJ2_DEC_STATE_ORIG_0];
p->state++;
p->dest = dest + 1;
}
}
/*
if (BCJ2_IS_32BIT_STREAM(p->state))
{
const Byte *cur = p->bufs[p->state];
if (cur == p->lims[p->state])
return SZ_OK;
p->bufs[p->state] = cur + 4;
{
UInt32 val;
Byte *dest;
SizeT rem;
p->ip += 4;
val = GetBe32(cur) - p->ip;
dest = p->dest;
rem = p->destLim - dest;
if (rem < 4)
{
SizeT i;
SetUi32(p->temp, val);
for (i = 0; i < rem; i++)
dest[i] = p->temp[i];
p->dest = dest + rem;
p->state = BCJ2_DEC_STATE_ORIG_0 + (unsigned)rem;
return SZ_OK;
}
SetUi32(dest, val);
p->temp[3] = (Byte)(val >> 24);
p->dest = dest + 4;
p->state = BCJ2_DEC_STATE_OK;
}
}
*/
for (;;)
{
if (BCJ2_IS_32BIT_STREAM(p->state))
p->state = BCJ2_DEC_STATE_OK;
else
{
if (p->range < kTopValue)
{
if (p->bufs[BCJ2_STREAM_RC] == p->lims[BCJ2_STREAM_RC])
{
p->state = BCJ2_STREAM_RC;
return SZ_OK;
}
p->range <<= 8;
p->code = (p->code << 8) | *(p->bufs[BCJ2_STREAM_RC])++;
}
{
const Byte *src = p->bufs[BCJ2_STREAM_MAIN];
const Byte *srcLim;
Byte *dest;
SizeT num = p->lims[BCJ2_STREAM_MAIN] - src;
if (num == 0)
{
p->state = BCJ2_STREAM_MAIN;
return SZ_OK;
}
dest = p->dest;
if (num > (SizeT)(p->destLim - dest))
{
num = p->destLim - dest;
if (num == 0)
{
p->state = BCJ2_DEC_STATE_ORIG;
return SZ_OK;
}
}
srcLim = src + num;
if (p->temp[3] == 0x0F && (src[0] & 0xF0) == 0x80)
*dest = src[0];
else for (;;)
{
Byte b = *src;
*dest = b;
if (b != 0x0F)
{
if ((b & 0xFE) == 0xE8)
break;
dest++;
if (++src != srcLim)
continue;
break;
}
dest++;
if (++src == srcLim)
break;
if ((*src & 0xF0) != 0x80)
continue;
*dest = *src;
break;
}
num = src - p->bufs[BCJ2_STREAM_MAIN];
if (src == srcLim)
{
p->temp[3] = src[-1];
p->bufs[BCJ2_STREAM_MAIN] = src;
p->ip += (UInt32)num;
p->dest += num;
p->state =
p->bufs[BCJ2_STREAM_MAIN] ==
p->lims[BCJ2_STREAM_MAIN] ?
(unsigned)BCJ2_STREAM_MAIN :
(unsigned)BCJ2_DEC_STATE_ORIG;
return SZ_OK;
}
{
UInt32 bound, ttt;
CProb *prob;
Byte b = src[0];
Byte prev = (Byte)(num == 0 ? p->temp[3] : src[-1]);
p->temp[3] = b;
p->bufs[BCJ2_STREAM_MAIN] = src + 1;
num++;
p->ip += (UInt32)num;
p->dest += num;
prob = p->probs + (unsigned)(b == 0xE8 ? 2 + (unsigned)prev : (b == 0xE9 ? 1 : 0));
_IF_BIT_0
{
_UPDATE_0
continue;
}
_UPDATE_1
}
}
}
{
UInt32 val;
unsigned cj = (p->temp[3] == 0xE8) ? BCJ2_STREAM_CALL : BCJ2_STREAM_JUMP;
const Byte *cur = p->bufs[cj];
Byte *dest;
SizeT rem;
if (cur == p->lims[cj])
{
p->state = cj;
break;
}
val = GetBe32(cur);
p->bufs[cj] = cur + 4;
p->ip += 4;
val -= p->ip;
dest = p->dest;
rem = p->destLim - dest;
if (rem < 4)
{
p->temp[0] = (Byte)val; if (rem > 0) dest[0] = (Byte)val; val >>= 8;
p->temp[1] = (Byte)val; if (rem > 1) dest[1] = (Byte)val; val >>= 8;
p->temp[2] = (Byte)val; if (rem > 2) dest[2] = (Byte)val; val >>= 8;
p->temp[3] = (Byte)val;
p->dest = dest + rem;
p->state = BCJ2_DEC_STATE_ORIG_0 + (unsigned)rem;
break;
}
SetUi32(dest, val);
p->temp[3] = (Byte)(val >> 24);
p->dest = dest + 4;
}
}
if (p->range < kTopValue && p->bufs[BCJ2_STREAM_RC] != p->lims[BCJ2_STREAM_RC])
{
p->range <<= 8;
p->code = (p->code << 8) | *(p->bufs[BCJ2_STREAM_RC])++;
}
return SZ_OK;
}
+146
View File
@@ -0,0 +1,146 @@
/* Bcj2.h -- BCJ2 Converter for x86 code
2014-11-10 : Igor Pavlov : Public domain */
#ifndef __BCJ2_H
#define __BCJ2_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define BCJ2_NUM_STREAMS 4
enum
{
BCJ2_STREAM_MAIN,
BCJ2_STREAM_CALL,
BCJ2_STREAM_JUMP,
BCJ2_STREAM_RC
};
enum
{
BCJ2_DEC_STATE_ORIG_0 = BCJ2_NUM_STREAMS,
BCJ2_DEC_STATE_ORIG_1,
BCJ2_DEC_STATE_ORIG_2,
BCJ2_DEC_STATE_ORIG_3,
BCJ2_DEC_STATE_ORIG,
BCJ2_DEC_STATE_OK
};
enum
{
BCJ2_ENC_STATE_ORIG = BCJ2_NUM_STREAMS,
BCJ2_ENC_STATE_OK
};
#define BCJ2_IS_32BIT_STREAM(s) ((s) == BCJ2_STREAM_CALL || (s) == BCJ2_STREAM_JUMP)
/*
CBcj2Dec / CBcj2Enc
bufs sizes:
BUF_SIZE(n) = lims[n] - bufs[n]
bufs sizes for BCJ2_STREAM_CALL and BCJ2_STREAM_JUMP must be mutliply of 4:
(BUF_SIZE(BCJ2_STREAM_CALL) & 3) == 0
(BUF_SIZE(BCJ2_STREAM_JUMP) & 3) == 0
*/
/*
CBcj2Dec:
dest is allowed to overlap with bufs[BCJ2_STREAM_MAIN], with the following conditions:
bufs[BCJ2_STREAM_MAIN] >= dest &&
bufs[BCJ2_STREAM_MAIN] - dest >= tempReserv +
BUF_SIZE(BCJ2_STREAM_CALL) +
BUF_SIZE(BCJ2_STREAM_JUMP)
tempReserv = 0 : for first call of Bcj2Dec_Decode
tempReserv = 4 : for any other calls of Bcj2Dec_Decode
overlap with offset = 1 is not allowed
*/
typedef struct
{
const Byte *bufs[BCJ2_NUM_STREAMS];
const Byte *lims[BCJ2_NUM_STREAMS];
Byte *dest;
const Byte *destLim;
unsigned state; /* BCJ2_STREAM_MAIN has more priority than BCJ2_STATE_ORIG */
UInt32 ip;
Byte temp[4];
UInt32 range;
UInt32 code;
UInt16 probs[2 + 256];
} CBcj2Dec;
void Bcj2Dec_Init(CBcj2Dec *p);
/* Returns: SZ_OK or SZ_ERROR_DATA */
SRes Bcj2Dec_Decode(CBcj2Dec *p);
#define Bcj2Dec_IsFinished(_p_) ((_p_)->code == 0)
typedef enum
{
BCJ2_ENC_FINISH_MODE_CONTINUE,
BCJ2_ENC_FINISH_MODE_END_BLOCK,
BCJ2_ENC_FINISH_MODE_END_STREAM
} EBcj2Enc_FinishMode;
typedef struct
{
Byte *bufs[BCJ2_NUM_STREAMS];
const Byte *lims[BCJ2_NUM_STREAMS];
const Byte *src;
const Byte *srcLim;
unsigned state;
EBcj2Enc_FinishMode finishMode;
Byte prevByte;
Byte cache;
UInt32 range;
UInt64 low;
UInt64 cacheSize;
UInt32 ip;
/* 32-bit ralative offset in JUMP/CALL commands is
- (mod 4 GB) in 32-bit mode
- signed Int32 in 64-bit mode
We use (mod 4 GB) check for fileSize.
Use fileSize up to 2 GB, if you want to support 32-bit and 64-bit code conversion. */
UInt32 fileIp;
UInt32 fileSize; /* (fileSize <= ((UInt32)1 << 31)), 0 means no_limit */
UInt32 relatLimit; /* (relatLimit <= ((UInt32)1 << 31)), 0 means desable_conversion */
UInt32 tempTarget;
unsigned tempPos;
Byte temp[4 * 2];
unsigned flushPos;
UInt16 probs[2 + 256];
} CBcj2Enc;
void Bcj2Enc_Init(CBcj2Enc *p);
void Bcj2Enc_Encode(CBcj2Enc *p);
#define Bcj2Enc_Get_InputData_Size(p) ((SizeT)((p)->srcLim - (p)->src) + (p)->tempPos)
#define Bcj2Enc_IsFinished(p) ((p)->flushPos == 5)
#define BCJ2_RELAT_LIMIT_NUM_BITS 26
#define BCJ2_RELAT_LIMIT ((UInt32)1 << BCJ2_RELAT_LIMIT_NUM_BITS)
/* limit for CBcj2Enc::fileSize variable */
#define BCJ2_FileSize_MAX ((UInt32)1 << 31)
EXTERN_C_END
#endif
+311
View File
@@ -0,0 +1,311 @@
/* Bcj2Enc.c -- BCJ2 Encoder (Converter for x86 code)
2019-02-02 : Igor Pavlov : Public domain */
#include "Precomp.h"
/* #define SHOW_STAT */
#ifdef SHOW_STAT
#include <stdio.h>
#define PRF(x) x
#else
#define PRF(x)
#endif
#include <string.h>
#include "Bcj2.h"
#include "CpuArch.h"
#define CProb UInt16
#define kTopValue ((UInt32)1 << 24)
#define kNumModelBits 11
#define kBitModelTotal (1 << kNumModelBits)
#define kNumMoveBits 5
void Bcj2Enc_Init(CBcj2Enc *p)
{
unsigned i;
p->state = BCJ2_ENC_STATE_OK;
p->finishMode = BCJ2_ENC_FINISH_MODE_CONTINUE;
p->prevByte = 0;
p->cache = 0;
p->range = 0xFFFFFFFF;
p->low = 0;
p->cacheSize = 1;
p->ip = 0;
p->fileIp = 0;
p->fileSize = 0;
p->relatLimit = BCJ2_RELAT_LIMIT;
p->tempPos = 0;
p->flushPos = 0;
for (i = 0; i < sizeof(p->probs) / sizeof(p->probs[0]); i++)
p->probs[i] = kBitModelTotal >> 1;
}
static BoolInt MY_FAST_CALL RangeEnc_ShiftLow(CBcj2Enc *p)
{
if ((UInt32)p->low < (UInt32)0xFF000000 || (UInt32)(p->low >> 32) != 0)
{
Byte *buf = p->bufs[BCJ2_STREAM_RC];
do
{
if (buf == p->lims[BCJ2_STREAM_RC])
{
p->state = BCJ2_STREAM_RC;
p->bufs[BCJ2_STREAM_RC] = buf;
return True;
}
*buf++ = (Byte)(p->cache + (Byte)(p->low >> 32));
p->cache = 0xFF;
}
while (--p->cacheSize);
p->bufs[BCJ2_STREAM_RC] = buf;
p->cache = (Byte)((UInt32)p->low >> 24);
}
p->cacheSize++;
p->low = (UInt32)p->low << 8;
return False;
}
static void Bcj2Enc_Encode_2(CBcj2Enc *p)
{
if (BCJ2_IS_32BIT_STREAM(p->state))
{
Byte *cur = p->bufs[p->state];
if (cur == p->lims[p->state])
return;
SetBe32(cur, p->tempTarget);
p->bufs[p->state] = cur + 4;
}
p->state = BCJ2_ENC_STATE_ORIG;
for (;;)
{
if (p->range < kTopValue)
{
if (RangeEnc_ShiftLow(p))
return;
p->range <<= 8;
}
{
{
const Byte *src = p->src;
const Byte *srcLim;
Byte *dest;
SizeT num = p->srcLim - src;
if (p->finishMode == BCJ2_ENC_FINISH_MODE_CONTINUE)
{
if (num <= 4)
return;
num -= 4;
}
else if (num == 0)
break;
dest = p->bufs[BCJ2_STREAM_MAIN];
if (num > (SizeT)(p->lims[BCJ2_STREAM_MAIN] - dest))
{
num = p->lims[BCJ2_STREAM_MAIN] - dest;
if (num == 0)
{
p->state = BCJ2_STREAM_MAIN;
return;
}
}
srcLim = src + num;
if (p->prevByte == 0x0F && (src[0] & 0xF0) == 0x80)
*dest = src[0];
else for (;;)
{
Byte b = *src;
*dest = b;
if (b != 0x0F)
{
if ((b & 0xFE) == 0xE8)
break;
dest++;
if (++src != srcLim)
continue;
break;
}
dest++;
if (++src == srcLim)
break;
if ((*src & 0xF0) != 0x80)
continue;
*dest = *src;
break;
}
num = src - p->src;
if (src == srcLim)
{
p->prevByte = src[-1];
p->bufs[BCJ2_STREAM_MAIN] = dest;
p->src = src;
p->ip += (UInt32)num;
continue;
}
{
Byte context = (Byte)(num == 0 ? p->prevByte : src[-1]);
BoolInt needConvert;
p->bufs[BCJ2_STREAM_MAIN] = dest + 1;
p->ip += (UInt32)num + 1;
src++;
needConvert = False;
if ((SizeT)(p->srcLim - src) >= 4)
{
UInt32 relatVal = GetUi32(src);
if ((p->fileSize == 0 || (UInt32)(p->ip + 4 + relatVal - p->fileIp) < p->fileSize)
&& ((relatVal + p->relatLimit) >> 1) < p->relatLimit)
needConvert = True;
}
{
UInt32 bound;
unsigned ttt;
Byte b = src[-1];
CProb *prob = p->probs + (unsigned)(b == 0xE8 ? 2 + (unsigned)context : (b == 0xE9 ? 1 : 0));
ttt = *prob;
bound = (p->range >> kNumModelBits) * ttt;
if (!needConvert)
{
p->range = bound;
*prob = (CProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
p->src = src;
p->prevByte = b;
continue;
}
p->low += bound;
p->range -= bound;
*prob = (CProb)(ttt - (ttt >> kNumMoveBits));
{
UInt32 relatVal = GetUi32(src);
UInt32 absVal;
p->ip += 4;
absVal = p->ip + relatVal;
p->prevByte = src[3];
src += 4;
p->src = src;
{
unsigned cj = (b == 0xE8) ? BCJ2_STREAM_CALL : BCJ2_STREAM_JUMP;
Byte *cur = p->bufs[cj];
if (cur == p->lims[cj])
{
p->state = cj;
p->tempTarget = absVal;
return;
}
SetBe32(cur, absVal);
p->bufs[cj] = cur + 4;
}
}
}
}
}
}
}
if (p->finishMode != BCJ2_ENC_FINISH_MODE_END_STREAM)
return;
for (; p->flushPos < 5; p->flushPos++)
if (RangeEnc_ShiftLow(p))
return;
p->state = BCJ2_ENC_STATE_OK;
}
void Bcj2Enc_Encode(CBcj2Enc *p)
{
PRF(printf("\n"));
PRF(printf("---- ip = %8d tempPos = %8d src = %8d\n", p->ip, p->tempPos, p->srcLim - p->src));
if (p->tempPos != 0)
{
unsigned extra = 0;
for (;;)
{
const Byte *src = p->src;
const Byte *srcLim = p->srcLim;
EBcj2Enc_FinishMode finishMode = p->finishMode;
p->src = p->temp;
p->srcLim = p->temp + p->tempPos;
if (src != srcLim)
p->finishMode = BCJ2_ENC_FINISH_MODE_CONTINUE;
PRF(printf(" ip = %8d tempPos = %8d src = %8d\n", p->ip, p->tempPos, p->srcLim - p->src));
Bcj2Enc_Encode_2(p);
{
unsigned num = (unsigned)(p->src - p->temp);
unsigned tempPos = p->tempPos - num;
unsigned i;
p->tempPos = tempPos;
for (i = 0; i < tempPos; i++)
p->temp[i] = p->temp[(size_t)i + num];
p->src = src;
p->srcLim = srcLim;
p->finishMode = finishMode;
if (p->state != BCJ2_ENC_STATE_ORIG || src == srcLim)
return;
if (extra >= tempPos)
{
p->src = src - tempPos;
p->tempPos = 0;
break;
}
p->temp[tempPos] = src[0];
p->tempPos = tempPos + 1;
p->src = src + 1;
extra++;
}
}
}
PRF(printf("++++ ip = %8d tempPos = %8d src = %8d\n", p->ip, p->tempPos, p->srcLim - p->src));
Bcj2Enc_Encode_2(p);
if (p->state == BCJ2_ENC_STATE_ORIG)
{
const Byte *src = p->src;
unsigned rem = (unsigned)(p->srcLim - src);
unsigned i;
for (i = 0; i < rem; i++)
p->temp[i] = src[i];
p->tempPos = rem;
p->src = src + rem;
}
}
+230
View File
@@ -0,0 +1,230 @@
/* Bra.c -- Converters for RISC code
2017-04-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "CpuArch.h"
#include "Bra.h"
SizeT ARM_Convert(Byte *data, SizeT size, UInt32 ip, int encoding)
{
Byte *p;
const Byte *lim;
size &= ~(size_t)3;
ip += 4;
p = data;
lim = data + size;
if (encoding)
for (;;)
{
for (;;)
{
if (p >= lim)
return p - data;
p += 4;
if (p[-1] == 0xEB)
break;
}
{
UInt32 v = GetUi32(p - 4);
v <<= 2;
v += ip + (UInt32)(p - data);
v >>= 2;
v &= 0x00FFFFFF;
v |= 0xEB000000;
SetUi32(p - 4, v);
}
}
for (;;)
{
for (;;)
{
if (p >= lim)
return p - data;
p += 4;
if (p[-1] == 0xEB)
break;
}
{
UInt32 v = GetUi32(p - 4);
v <<= 2;
v -= ip + (UInt32)(p - data);
v >>= 2;
v &= 0x00FFFFFF;
v |= 0xEB000000;
SetUi32(p - 4, v);
}
}
}
SizeT ARMT_Convert(Byte *data, SizeT size, UInt32 ip, int encoding)
{
Byte *p;
const Byte *lim;
size &= ~(size_t)1;
p = data;
lim = data + size - 4;
if (encoding)
for (;;)
{
UInt32 b1;
for (;;)
{
UInt32 b3;
if (p > lim)
return p - data;
b1 = p[1];
b3 = p[3];
p += 2;
b1 ^= 8;
if ((b3 & b1) >= 0xF8)
break;
}
{
UInt32 v =
((UInt32)b1 << 19)
+ (((UInt32)p[1] & 0x7) << 8)
+ (((UInt32)p[-2] << 11))
+ (p[0]);
p += 2;
{
UInt32 cur = (ip + (UInt32)(p - data)) >> 1;
v += cur;
}
p[-4] = (Byte)(v >> 11);
p[-3] = (Byte)(0xF0 | ((v >> 19) & 0x7));
p[-2] = (Byte)v;
p[-1] = (Byte)(0xF8 | (v >> 8));
}
}
for (;;)
{
UInt32 b1;
for (;;)
{
UInt32 b3;
if (p > lim)
return p - data;
b1 = p[1];
b3 = p[3];
p += 2;
b1 ^= 8;
if ((b3 & b1) >= 0xF8)
break;
}
{
UInt32 v =
((UInt32)b1 << 19)
+ (((UInt32)p[1] & 0x7) << 8)
+ (((UInt32)p[-2] << 11))
+ (p[0]);
p += 2;
{
UInt32 cur = (ip + (UInt32)(p - data)) >> 1;
v -= cur;
}
/*
SetUi16(p - 4, (UInt16)(((v >> 11) & 0x7FF) | 0xF000));
SetUi16(p - 2, (UInt16)(v | 0xF800));
*/
p[-4] = (Byte)(v >> 11);
p[-3] = (Byte)(0xF0 | ((v >> 19) & 0x7));
p[-2] = (Byte)v;
p[-1] = (Byte)(0xF8 | (v >> 8));
}
}
}
SizeT PPC_Convert(Byte *data, SizeT size, UInt32 ip, int encoding)
{
Byte *p;
const Byte *lim;
size &= ~(size_t)3;
ip -= 4;
p = data;
lim = data + size;
for (;;)
{
for (;;)
{
if (p >= lim)
return p - data;
p += 4;
/* if ((v & 0xFC000003) == 0x48000001) */
if ((p[-4] & 0xFC) == 0x48 && (p[-1] & 3) == 1)
break;
}
{
UInt32 v = GetBe32(p - 4);
if (encoding)
v += ip + (UInt32)(p - data);
else
v -= ip + (UInt32)(p - data);
v &= 0x03FFFFFF;
v |= 0x48000000;
SetBe32(p - 4, v);
}
}
}
SizeT SPARC_Convert(Byte *data, SizeT size, UInt32 ip, int encoding)
{
Byte *p;
const Byte *lim;
size &= ~(size_t)3;
ip -= 4;
p = data;
lim = data + size;
for (;;)
{
for (;;)
{
if (p >= lim)
return p - data;
/*
v = GetBe32(p);
p += 4;
m = v + ((UInt32)5 << 29);
m ^= (UInt32)7 << 29;
m += (UInt32)1 << 22;
if ((m & ((UInt32)0x1FF << 23)) == 0)
break;
*/
p += 4;
if ((p[-4] == 0x40 && (p[-3] & 0xC0) == 0) ||
(p[-4] == 0x7F && (p[-3] >= 0xC0)))
break;
}
{
UInt32 v = GetBe32(p - 4);
v <<= 2;
if (encoding)
v += ip + (UInt32)(p - data);
else
v -= ip + (UInt32)(p - data);
v &= 0x01FFFFFF;
v -= (UInt32)1 << 24;
v ^= 0xFF000000;
v >>= 2;
v |= 0x40000000;
SetBe32(p - 4, v);
}
}
}
+64
View File
@@ -0,0 +1,64 @@
/* Bra.h -- Branch converters for executables
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __BRA_H
#define __BRA_H
#include "7zTypes.h"
EXTERN_C_BEGIN
/*
These functions convert relative addresses to absolute addresses
in CALL instructions to increase the compression ratio.
In:
data - data buffer
size - size of data
ip - current virtual Instruction Pinter (IP) value
state - state variable for x86 converter
encoding - 0 (for decoding), 1 (for encoding)
Out:
state - state variable for x86 converter
Returns:
The number of processed bytes. If you call these functions with multiple calls,
you must start next call with first byte after block of processed bytes.
Type Endian Alignment LookAhead
x86 little 1 4
ARMT little 2 2
ARM little 4 0
PPC big 4 0
SPARC big 4 0
IA64 little 16 0
size must be >= Alignment + LookAhead, if it's not last block.
If (size < Alignment + LookAhead), converter returns 0.
Example:
UInt32 ip = 0;
for ()
{
; size must be >= Alignment + LookAhead, if it's not last block
SizeT processed = Convert(data, size, ip, 1);
data += processed;
size -= processed;
ip += processed;
}
*/
#define x86_Convert_Init(state) { state = 0; }
SizeT x86_Convert(Byte *data, SizeT size, UInt32 ip, UInt32 *state, int encoding);
SizeT ARM_Convert(Byte *data, SizeT size, UInt32 ip, int encoding);
SizeT ARMT_Convert(Byte *data, SizeT size, UInt32 ip, int encoding);
SizeT PPC_Convert(Byte *data, SizeT size, UInt32 ip, int encoding);
SizeT SPARC_Convert(Byte *data, SizeT size, UInt32 ip, int encoding);
SizeT IA64_Convert(Byte *data, SizeT size, UInt32 ip, int encoding);
EXTERN_C_END
#endif
+82
View File
@@ -0,0 +1,82 @@
/* Bra86.c -- Converter for x86 code (BCJ)
2017-04-03 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "Bra.h"
#define Test86MSByte(b) ((((b) + 1) & 0xFE) == 0)
SizeT x86_Convert(Byte *data, SizeT size, UInt32 ip, UInt32 *state, int encoding)
{
SizeT pos = 0;
UInt32 mask = *state & 7;
if (size < 5)
return 0;
size -= 4;
ip += 5;
for (;;)
{
Byte *p = data + pos;
const Byte *limit = data + size;
for (; p < limit; p++)
if ((*p & 0xFE) == 0xE8)
break;
{
SizeT d = (SizeT)(p - data - pos);
pos = (SizeT)(p - data);
if (p >= limit)
{
*state = (d > 2 ? 0 : mask >> (unsigned)d);
return pos;
}
if (d > 2)
mask = 0;
else
{
mask >>= (unsigned)d;
if (mask != 0 && (mask > 4 || mask == 3 || Test86MSByte(p[(size_t)(mask >> 1) + 1])))
{
mask = (mask >> 1) | 4;
pos++;
continue;
}
}
}
if (Test86MSByte(p[4]))
{
UInt32 v = ((UInt32)p[4] << 24) | ((UInt32)p[3] << 16) | ((UInt32)p[2] << 8) | ((UInt32)p[1]);
UInt32 cur = ip + (UInt32)pos;
pos += 5;
if (encoding)
v += cur;
else
v -= cur;
if (mask != 0)
{
unsigned sh = (mask & 6) << 2;
if (Test86MSByte((Byte)(v >> sh)))
{
v ^= (((UInt32)0x100 << sh) - 1);
if (encoding)
v += cur;
else
v -= cur;
}
mask = 0;
}
p[1] = (Byte)v;
p[2] = (Byte)(v >> 8);
p[3] = (Byte)(v >> 16);
p[4] = (Byte)(0 - ((v >> 24) & 1));
}
else
{
mask = (mask >> 1) | 4;
pos++;
}
}
}
+53
View File
@@ -0,0 +1,53 @@
/* BraIA64.c -- Converter for IA-64 code
2017-01-26 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "CpuArch.h"
#include "Bra.h"
SizeT IA64_Convert(Byte *data, SizeT size, UInt32 ip, int encoding)
{
SizeT i;
if (size < 16)
return 0;
size -= 16;
i = 0;
do
{
unsigned m = ((UInt32)0x334B0000 >> (data[i] & 0x1E)) & 3;
if (m)
{
m++;
do
{
Byte *p = data + (i + (size_t)m * 5 - 8);
if (((p[3] >> m) & 15) == 5
&& (((p[-1] | ((UInt32)p[0] << 8)) >> m) & 0x70) == 0)
{
unsigned raw = GetUi32(p);
unsigned v = raw >> m;
v = (v & 0xFFFFF) | ((v & (1 << 23)) >> 3);
v <<= 4;
if (encoding)
v += ip + (UInt32)i;
else
v -= ip + (UInt32)i;
v >>= 4;
v &= 0x1FFFFF;
v += 0x700000;
v &= 0x8FFFFF;
raw &= ~((UInt32)0x8FFFFF << m);
raw |= (v << m);
SetUi32(p, raw);
}
}
while (++m <= 4);
}
i += 16;
}
while (i <= size);
return i;
}
+33
View File
@@ -0,0 +1,33 @@
/* Compiler.h
2017-04-03 : Igor Pavlov : Public domain */
#ifndef __7Z_COMPILER_H
#define __7Z_COMPILER_H
#ifdef _MSC_VER
#ifdef UNDER_CE
#define RPC_NO_WINDOWS_H
/* #pragma warning(disable : 4115) // '_RPC_ASYNC_STATE' : named type definition in parentheses */
#pragma warning(disable : 4201) // nonstandard extension used : nameless struct/union
#pragma warning(disable : 4214) // nonstandard extension used : bit field types other than int
#endif
#if _MSC_VER >= 1300
#pragma warning(disable : 4996) // This function or variable may be unsafe
#else
#pragma warning(disable : 4511) // copy constructor could not be generated
#pragma warning(disable : 4512) // assignment operator could not be generated
#pragma warning(disable : 4514) // unreferenced inline function has been removed
#pragma warning(disable : 4702) // unreachable code
#pragma warning(disable : 4710) // not inlined
#pragma warning(disable : 4714) // function marked as __forceinline not inlined
#pragma warning(disable : 4786) // identifier was truncated to '255' characters in the debug information
#endif
#endif
#define UNUSED_VAR(x) (void)x;
/* #define UNUSED_VAR(x) x=x; */
#endif
+218
View File
@@ -0,0 +1,218 @@
/* CpuArch.c -- CPU specific code
2018-02-18: Igor Pavlov : Public domain */
#include "Precomp.h"
#include "CpuArch.h"
#ifdef MY_CPU_X86_OR_AMD64
#if (defined(_MSC_VER) && !defined(MY_CPU_AMD64)) || defined(__GNUC__)
#define USE_ASM
#endif
#if !defined(USE_ASM) && _MSC_VER >= 1500
#include <intrin.h>
#endif
#if defined(USE_ASM) && !defined(MY_CPU_AMD64)
static UInt32 CheckFlag(UInt32 flag)
{
#ifdef _MSC_VER
__asm pushfd;
__asm pop EAX;
__asm mov EDX, EAX;
__asm xor EAX, flag;
__asm push EAX;
__asm popfd;
__asm pushfd;
__asm pop EAX;
__asm xor EAX, EDX;
__asm push EDX;
__asm popfd;
__asm and flag, EAX;
#else
__asm__ __volatile__ (
"pushf\n\t"
"pop %%EAX\n\t"
"movl %%EAX,%%EDX\n\t"
"xorl %0,%%EAX\n\t"
"push %%EAX\n\t"
"popf\n\t"
"pushf\n\t"
"pop %%EAX\n\t"
"xorl %%EDX,%%EAX\n\t"
"push %%EDX\n\t"
"popf\n\t"
"andl %%EAX, %0\n\t":
"=c" (flag) : "c" (flag) :
"%eax", "%edx");
#endif
return flag;
}
#define CHECK_CPUID_IS_SUPPORTED if (CheckFlag(1 << 18) == 0 || CheckFlag(1 << 21) == 0) return False;
#else
#define CHECK_CPUID_IS_SUPPORTED
#endif
void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d)
{
#ifdef USE_ASM
#ifdef _MSC_VER
UInt32 a2, b2, c2, d2;
__asm xor EBX, EBX;
__asm xor ECX, ECX;
__asm xor EDX, EDX;
__asm mov EAX, function;
__asm cpuid;
__asm mov a2, EAX;
__asm mov b2, EBX;
__asm mov c2, ECX;
__asm mov d2, EDX;
*a = a2;
*b = b2;
*c = c2;
*d = d2;
#else
__asm__ __volatile__ (
#if defined(MY_CPU_AMD64) && defined(__PIC__)
"mov %%rbx, %%rdi;"
"cpuid;"
"xchg %%rbx, %%rdi;"
: "=a" (*a) ,
"=D" (*b) ,
#elif defined(MY_CPU_X86) && defined(__PIC__)
"mov %%ebx, %%edi;"
"cpuid;"
"xchgl %%ebx, %%edi;"
: "=a" (*a) ,
"=D" (*b) ,
#else
"cpuid"
: "=a" (*a) ,
"=b" (*b) ,
#endif
"=c" (*c) ,
"=d" (*d)
: "0" (function)) ;
#endif
#else
int CPUInfo[4];
__cpuid(CPUInfo, function);
*a = CPUInfo[0];
*b = CPUInfo[1];
*c = CPUInfo[2];
*d = CPUInfo[3];
#endif
}
BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p)
{
CHECK_CPUID_IS_SUPPORTED
MyCPUID(0, &p->maxFunc, &p->vendor[0], &p->vendor[2], &p->vendor[1]);
MyCPUID(1, &p->ver, &p->b, &p->c, &p->d);
return True;
}
static const UInt32 kVendors[][3] =
{
{ 0x756E6547, 0x49656E69, 0x6C65746E},
{ 0x68747541, 0x69746E65, 0x444D4163},
{ 0x746E6543, 0x48727561, 0x736C7561}
};
int x86cpuid_GetFirm(const Cx86cpuid *p)
{
unsigned i;
for (i = 0; i < sizeof(kVendors) / sizeof(kVendors[i]); i++)
{
const UInt32 *v = kVendors[i];
if (v[0] == p->vendor[0] &&
v[1] == p->vendor[1] &&
v[2] == p->vendor[2])
return (int)i;
}
return -1;
}
BoolInt CPU_Is_InOrder()
{
Cx86cpuid p;
int firm;
UInt32 family, model;
if (!x86cpuid_CheckAndRead(&p))
return True;
family = x86cpuid_GetFamily(p.ver);
model = x86cpuid_GetModel(p.ver);
firm = x86cpuid_GetFirm(&p);
switch (firm)
{
case CPU_FIRM_INTEL: return (family < 6 || (family == 6 && (
/* In-Order Atom CPU */
model == 0x1C /* 45 nm, N4xx, D4xx, N5xx, D5xx, 230, 330 */
|| model == 0x26 /* 45 nm, Z6xx */
|| model == 0x27 /* 32 nm, Z2460 */
|| model == 0x35 /* 32 nm, Z2760 */
|| model == 0x36 /* 32 nm, N2xxx, D2xxx */
)));
case CPU_FIRM_AMD: return (family < 5 || (family == 5 && (model < 6 || model == 0xA)));
case CPU_FIRM_VIA: return (family < 6 || (family == 6 && model < 0xF));
}
return True;
}
#if !defined(MY_CPU_AMD64) && defined(_WIN32)
#include <windows.h>
static BoolInt CPU_Sys_Is_SSE_Supported()
{
OSVERSIONINFO vi;
vi.dwOSVersionInfoSize = sizeof(vi);
if (!GetVersionEx(&vi))
return False;
return (vi.dwMajorVersion >= 5);
}
#define CHECK_SYS_SSE_SUPPORT if (!CPU_Sys_Is_SSE_Supported()) return False;
#else
#define CHECK_SYS_SSE_SUPPORT
#endif
BoolInt CPU_Is_Aes_Supported()
{
Cx86cpuid p;
CHECK_SYS_SSE_SUPPORT
if (!x86cpuid_CheckAndRead(&p))
return False;
return (p.c >> 25) & 1;
}
BoolInt CPU_IsSupported_PageGB()
{
Cx86cpuid cpuid;
if (!x86cpuid_CheckAndRead(&cpuid))
return False;
{
UInt32 d[4] = { 0 };
MyCPUID(0x80000000, &d[0], &d[1], &d[2], &d[3]);
if (d[0] < 0x80000001)
return False;
}
{
UInt32 d[4] = { 0 };
MyCPUID(0x80000001, &d[0], &d[1], &d[2], &d[3]);
return (d[3] >> 26) & 1;
}
}
#endif
+336
View File
@@ -0,0 +1,336 @@
/* CpuArch.h -- CPU specific code
2018-02-18 : Igor Pavlov : Public domain */
#ifndef __CPU_ARCH_H
#define __CPU_ARCH_H
#include "7zTypes.h"
EXTERN_C_BEGIN
/*
MY_CPU_LE means that CPU is LITTLE ENDIAN.
MY_CPU_BE means that CPU is BIG ENDIAN.
If MY_CPU_LE and MY_CPU_BE are not defined, we don't know about ENDIANNESS of platform.
MY_CPU_LE_UNALIGN means that CPU is LITTLE ENDIAN and CPU supports unaligned memory accesses.
*/
#if defined(_M_X64) \
|| defined(_M_AMD64) \
|| defined(__x86_64__) \
|| defined(__AMD64__) \
|| defined(__amd64__)
#define MY_CPU_AMD64
#ifdef __ILP32__
#define MY_CPU_NAME "x32"
#else
#define MY_CPU_NAME "x64"
#endif
#define MY_CPU_64BIT
#endif
#if defined(_M_IX86) \
|| defined(__i386__)
#define MY_CPU_X86
#define MY_CPU_NAME "x86"
#define MY_CPU_32BIT
#endif
#if defined(_M_ARM64) \
|| defined(__AARCH64EL__) \
|| defined(__AARCH64EB__) \
|| defined(__aarch64__)
#define MY_CPU_ARM64
#define MY_CPU_NAME "arm64"
#define MY_CPU_64BIT
#endif
#if defined(_M_ARM) \
|| defined(_M_ARM_NT) \
|| defined(_M_ARMT) \
|| defined(__arm__) \
|| defined(__thumb__) \
|| defined(__ARMEL__) \
|| defined(__ARMEB__) \
|| defined(__THUMBEL__) \
|| defined(__THUMBEB__)
#define MY_CPU_ARM
#define MY_CPU_NAME "arm"
#define MY_CPU_32BIT
#endif
#if defined(_M_IA64) \
|| defined(__ia64__)
#define MY_CPU_IA64
#define MY_CPU_NAME "ia64"
#define MY_CPU_64BIT
#endif
#if defined(__mips64) \
|| defined(__mips64__) \
|| (defined(__mips) && (__mips == 64 || __mips == 4 || __mips == 3))
#define MY_CPU_NAME "mips64"
#define MY_CPU_64BIT
#elif defined(__mips__)
#define MY_CPU_NAME "mips"
/* #define MY_CPU_32BIT */
#endif
#if defined(__ppc64__) \
|| defined(__powerpc64__)
#ifdef __ILP32__
#define MY_CPU_NAME "ppc64-32"
#else
#define MY_CPU_NAME "ppc64"
#endif
#define MY_CPU_64BIT
#elif defined(__ppc__) \
|| defined(__powerpc__)
#define MY_CPU_NAME "ppc"
#define MY_CPU_32BIT
#endif
#if defined(__sparc64__)
#define MY_CPU_NAME "sparc64"
#define MY_CPU_64BIT
#elif defined(__sparc__)
#define MY_CPU_NAME "sparc"
/* #define MY_CPU_32BIT */
#endif
#if defined(MY_CPU_X86) || defined(MY_CPU_AMD64)
#define MY_CPU_X86_OR_AMD64
#endif
#ifdef _WIN32
#ifdef MY_CPU_ARM
#define MY_CPU_ARM_LE
#endif
#ifdef MY_CPU_ARM64
#define MY_CPU_ARM64_LE
#endif
#ifdef _M_IA64
#define MY_CPU_IA64_LE
#endif
#endif
#if defined(MY_CPU_X86_OR_AMD64) \
|| defined(MY_CPU_ARM_LE) \
|| defined(MY_CPU_ARM64_LE) \
|| defined(MY_CPU_IA64_LE) \
|| defined(__LITTLE_ENDIAN__) \
|| defined(__ARMEL__) \
|| defined(__THUMBEL__) \
|| defined(__AARCH64EL__) \
|| defined(__MIPSEL__) \
|| defined(__MIPSEL) \
|| defined(_MIPSEL) \
|| defined(__BFIN__) \
|| (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__))
#define MY_CPU_LE
#endif
#if defined(__BIG_ENDIAN__) \
|| defined(__ARMEB__) \
|| defined(__THUMBEB__) \
|| defined(__AARCH64EB__) \
|| defined(__MIPSEB__) \
|| defined(__MIPSEB) \
|| defined(_MIPSEB) \
|| defined(__m68k__) \
|| defined(__s390__) \
|| defined(__s390x__) \
|| defined(__zarch__) \
|| (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__))
#define MY_CPU_BE
#endif
#if defined(MY_CPU_LE) && defined(MY_CPU_BE)
#error Stop_Compiling_Bad_Endian
#endif
#if defined(MY_CPU_32BIT) && defined(MY_CPU_64BIT)
#error Stop_Compiling_Bad_32_64_BIT
#endif
#ifndef MY_CPU_NAME
#ifdef MY_CPU_LE
#define MY_CPU_NAME "LE"
#elif defined(MY_CPU_BE)
#define MY_CPU_NAME "BE"
#else
/*
#define MY_CPU_NAME ""
*/
#endif
#endif
#ifdef MY_CPU_LE
#if defined(MY_CPU_X86_OR_AMD64) \
|| defined(MY_CPU_ARM64) \
|| defined(__ARM_FEATURE_UNALIGNED)
#define MY_CPU_LE_UNALIGN
#endif
#endif
#ifdef MY_CPU_LE_UNALIGN
#define GetUi16(p) (*(const UInt16 *)(const void *)(p))
#define GetUi32(p) (*(const UInt32 *)(const void *)(p))
#define GetUi64(p) (*(const UInt64 *)(const void *)(p))
#define SetUi16(p, v) { *(UInt16 *)(p) = (v); }
#define SetUi32(p, v) { *(UInt32 *)(p) = (v); }
#define SetUi64(p, v) { *(UInt64 *)(p) = (v); }
#else
#define GetUi16(p) ( (UInt16) ( \
((const Byte *)(p))[0] | \
((UInt16)((const Byte *)(p))[1] << 8) ))
#define GetUi32(p) ( \
((const Byte *)(p))[0] | \
((UInt32)((const Byte *)(p))[1] << 8) | \
((UInt32)((const Byte *)(p))[2] << 16) | \
((UInt32)((const Byte *)(p))[3] << 24))
#define GetUi64(p) (GetUi32(p) | ((UInt64)GetUi32(((const Byte *)(p)) + 4) << 32))
#define SetUi16(p, v) { Byte *_ppp_ = (Byte *)(p); UInt32 _vvv_ = (v); \
_ppp_[0] = (Byte)_vvv_; \
_ppp_[1] = (Byte)(_vvv_ >> 8); }
#define SetUi32(p, v) { Byte *_ppp_ = (Byte *)(p); UInt32 _vvv_ = (v); \
_ppp_[0] = (Byte)_vvv_; \
_ppp_[1] = (Byte)(_vvv_ >> 8); \
_ppp_[2] = (Byte)(_vvv_ >> 16); \
_ppp_[3] = (Byte)(_vvv_ >> 24); }
#define SetUi64(p, v) { Byte *_ppp2_ = (Byte *)(p); UInt64 _vvv2_ = (v); \
SetUi32(_ppp2_ , (UInt32)_vvv2_); \
SetUi32(_ppp2_ + 4, (UInt32)(_vvv2_ >> 32)); }
#endif
#ifdef __has_builtin
#define MY__has_builtin(x) __has_builtin(x)
#else
#define MY__has_builtin(x) 0
#endif
#if defined(MY_CPU_LE_UNALIGN) && /* defined(_WIN64) && */ (_MSC_VER >= 1300)
/* Note: we use bswap instruction, that is unsupported in 386 cpu */
#include <stdlib.h>
#pragma intrinsic(_byteswap_ushort)
#pragma intrinsic(_byteswap_ulong)
#pragma intrinsic(_byteswap_uint64)
/* #define GetBe16(p) _byteswap_ushort(*(const UInt16 *)(const Byte *)(p)) */
#define GetBe32(p) _byteswap_ulong(*(const UInt32 *)(const Byte *)(p))
#define GetBe64(p) _byteswap_uint64(*(const UInt64 *)(const Byte *)(p))
#define SetBe32(p, v) (*(UInt32 *)(void *)(p)) = _byteswap_ulong(v)
#elif defined(MY_CPU_LE_UNALIGN) && ( \
(defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3))) \
|| (defined(__clang__) && MY__has_builtin(__builtin_bswap16)) )
/* #define GetBe16(p) __builtin_bswap16(*(const UInt16 *)(const Byte *)(p)) */
#define GetBe32(p) __builtin_bswap32(*(const UInt32 *)(const Byte *)(p))
#define GetBe64(p) __builtin_bswap64(*(const UInt64 *)(const Byte *)(p))
#define SetBe32(p, v) (*(UInt32 *)(void *)(p)) = __builtin_bswap32(v)
#else
#define GetBe32(p) ( \
((UInt32)((const Byte *)(p))[0] << 24) | \
((UInt32)((const Byte *)(p))[1] << 16) | \
((UInt32)((const Byte *)(p))[2] << 8) | \
((const Byte *)(p))[3] )
#define GetBe64(p) (((UInt64)GetBe32(p) << 32) | GetBe32(((const Byte *)(p)) + 4))
#define SetBe32(p, v) { Byte *_ppp_ = (Byte *)(p); UInt32 _vvv_ = (v); \
_ppp_[0] = (Byte)(_vvv_ >> 24); \
_ppp_[1] = (Byte)(_vvv_ >> 16); \
_ppp_[2] = (Byte)(_vvv_ >> 8); \
_ppp_[3] = (Byte)_vvv_; }
#endif
#ifndef GetBe16
#define GetBe16(p) ( (UInt16) ( \
((UInt16)((const Byte *)(p))[0] << 8) | \
((const Byte *)(p))[1] ))
#endif
#ifdef MY_CPU_X86_OR_AMD64
typedef struct
{
UInt32 maxFunc;
UInt32 vendor[3];
UInt32 ver;
UInt32 b;
UInt32 c;
UInt32 d;
} Cx86cpuid;
enum
{
CPU_FIRM_INTEL,
CPU_FIRM_AMD,
CPU_FIRM_VIA
};
void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d);
BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
int x86cpuid_GetFirm(const Cx86cpuid *p);
#define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) & 0xF))
#define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) & 0xF))
#define x86cpuid_GetStepping(ver) (ver & 0xF)
BoolInt CPU_Is_InOrder();
BoolInt CPU_Is_Aes_Supported();
BoolInt CPU_IsSupported_PageGB();
#endif
EXTERN_C_END
#endif
+64
View File
@@ -0,0 +1,64 @@
/* Delta.c -- Delta converter
2009-05-26 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "Delta.h"
void Delta_Init(Byte *state)
{
unsigned i;
for (i = 0; i < DELTA_STATE_SIZE; i++)
state[i] = 0;
}
static void MyMemCpy(Byte *dest, const Byte *src, unsigned size)
{
unsigned i;
for (i = 0; i < size; i++)
dest[i] = src[i];
}
void Delta_Encode(Byte *state, unsigned delta, Byte *data, SizeT size)
{
Byte buf[DELTA_STATE_SIZE];
unsigned j = 0;
MyMemCpy(buf, state, delta);
{
SizeT i;
for (i = 0; i < size;)
{
for (j = 0; j < delta && i < size; i++, j++)
{
Byte b = data[i];
data[i] = (Byte)(b - buf[j]);
buf[j] = b;
}
}
}
if (j == delta)
j = 0;
MyMemCpy(state, buf + j, delta - j);
MyMemCpy(state + delta - j, buf, j);
}
void Delta_Decode(Byte *state, unsigned delta, Byte *data, SizeT size)
{
Byte buf[DELTA_STATE_SIZE];
unsigned j = 0;
MyMemCpy(buf, state, delta);
{
SizeT i;
for (i = 0; i < size;)
{
for (j = 0; j < delta && i < size; i++, j++)
{
buf[j] = data[i] = (Byte)(buf[j] + data[i]);
}
}
}
if (j == delta)
j = 0;
MyMemCpy(state, buf + j, delta - j);
MyMemCpy(state + delta - j, buf, j);
}
+19
View File
@@ -0,0 +1,19 @@
/* Delta.h -- Delta converter
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __DELTA_H
#define __DELTA_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define DELTA_STATE_SIZE 256
void Delta_Init(Byte *state);
void Delta_Encode(Byte *state, unsigned delta, Byte *data, SizeT size);
void Delta_Decode(Byte *state, unsigned delta, Byte *data, SizeT size);
EXTERN_C_END
#endif
+108
View File
@@ -0,0 +1,108 @@
/* DllSecur.c -- DLL loading security
2018-02-21 : Igor Pavlov : Public domain */
#include "Precomp.h"
#ifdef _WIN32
#include <windows.h>
#include "DllSecur.h"
#ifndef UNDER_CE
typedef BOOL (WINAPI *Func_SetDefaultDllDirectories)(DWORD DirectoryFlags);
#define MY_LOAD_LIBRARY_SEARCH_USER_DIRS 0x400
#define MY_LOAD_LIBRARY_SEARCH_SYSTEM32 0x800
static const char * const g_Dlls =
#ifndef _CONSOLE
"UXTHEME\0"
#endif
"USERENV\0"
"SETUPAPI\0"
"APPHELP\0"
"PROPSYS\0"
"DWMAPI\0"
"CRYPTBASE\0"
"OLEACC\0"
"CLBCATQ\0"
"VERSION\0"
;
#endif
void My_SetDefaultDllDirectories()
{
#ifndef UNDER_CE
OSVERSIONINFO vi;
vi.dwOSVersionInfoSize = sizeof(vi);
GetVersionEx(&vi);
if (!GetVersionEx(&vi) || vi.dwMajorVersion != 6 || vi.dwMinorVersion != 0)
{
Func_SetDefaultDllDirectories setDllDirs = (Func_SetDefaultDllDirectories)
GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "SetDefaultDllDirectories");
if (setDllDirs)
if (setDllDirs(MY_LOAD_LIBRARY_SEARCH_SYSTEM32 | MY_LOAD_LIBRARY_SEARCH_USER_DIRS))
return;
}
#endif
}
void LoadSecurityDlls()
{
#ifndef UNDER_CE
wchar_t buf[MAX_PATH + 100];
{
// at Vista (ver 6.0) : CoCreateInstance(CLSID_ShellLink, ...) doesn't work after SetDefaultDllDirectories() : Check it ???
OSVERSIONINFO vi;
vi.dwOSVersionInfoSize = sizeof(vi);
if (!GetVersionEx(&vi) || vi.dwMajorVersion != 6 || vi.dwMinorVersion != 0)
{
Func_SetDefaultDllDirectories setDllDirs = (Func_SetDefaultDllDirectories)
GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "SetDefaultDllDirectories");
if (setDllDirs)
if (setDllDirs(MY_LOAD_LIBRARY_SEARCH_SYSTEM32 | MY_LOAD_LIBRARY_SEARCH_USER_DIRS))
return;
}
}
{
unsigned len = GetSystemDirectoryW(buf, MAX_PATH + 2);
if (len == 0 || len > MAX_PATH)
return;
}
{
const char *dll;
unsigned pos = (unsigned)lstrlenW(buf);
if (buf[pos - 1] != '\\')
buf[pos++] = '\\';
for (dll = g_Dlls; dll[0] != 0;)
{
unsigned k = 0;
for (;;)
{
char c = *dll++;
buf[pos + k] = (Byte)c;
k++;
if (c == 0)
break;
}
lstrcatW(buf, L".dll");
LoadLibraryExW(buf, NULL, LOAD_WITH_ALTERED_SEARCH_PATH);
}
}
#endif
}
#endif
+20
View File
@@ -0,0 +1,20 @@
/* DllSecur.h -- DLL loading for security
2018-02-19 : Igor Pavlov : Public domain */
#ifndef __DLL_SECUR_H
#define __DLL_SECUR_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#ifdef _WIN32
void My_SetDefaultDllDirectories();
void LoadSecurityDlls();
#endif
EXTERN_C_END
#endif
+1127
View File
File diff suppressed because it is too large Load Diff
+121
View File
@@ -0,0 +1,121 @@
/* LzFind.h -- Match finder for LZ algorithms
2017-06-10 : Igor Pavlov : Public domain */
#ifndef __LZ_FIND_H
#define __LZ_FIND_H
#include "7zTypes.h"
EXTERN_C_BEGIN
typedef UInt32 CLzRef;
typedef struct _CMatchFinder
{
Byte *buffer;
UInt32 pos;
UInt32 posLimit;
UInt32 streamPos;
UInt32 lenLimit;
UInt32 cyclicBufferPos;
UInt32 cyclicBufferSize; /* it must be = (historySize + 1) */
Byte streamEndWasReached;
Byte btMode;
Byte bigHash;
Byte directInput;
UInt32 matchMaxLen;
CLzRef *hash;
CLzRef *son;
UInt32 hashMask;
UInt32 cutValue;
Byte *bufferBase;
ISeqInStream *stream;
UInt32 blockSize;
UInt32 keepSizeBefore;
UInt32 keepSizeAfter;
UInt32 numHashBytes;
size_t directInputRem;
UInt32 historySize;
UInt32 fixedHashSize;
UInt32 hashSizeSum;
SRes result;
UInt32 crc[256];
size_t numRefs;
UInt64 expectedDataSize;
} CMatchFinder;
#define Inline_MatchFinder_GetPointerToCurrentPos(p) ((p)->buffer)
#define Inline_MatchFinder_GetNumAvailableBytes(p) ((p)->streamPos - (p)->pos)
#define Inline_MatchFinder_IsFinishedOK(p) \
((p)->streamEndWasReached \
&& (p)->streamPos == (p)->pos \
&& (!(p)->directInput || (p)->directInputRem == 0))
int MatchFinder_NeedMove(CMatchFinder *p);
Byte *MatchFinder_GetPointerToCurrentPos(CMatchFinder *p);
void MatchFinder_MoveBlock(CMatchFinder *p);
void MatchFinder_ReadIfRequired(CMatchFinder *p);
void MatchFinder_Construct(CMatchFinder *p);
/* Conditions:
historySize <= 3 GB
keepAddBufferBefore + matchMaxLen + keepAddBufferAfter < 511MB
*/
int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
UInt32 keepAddBufferBefore, UInt32 matchMaxLen, UInt32 keepAddBufferAfter,
ISzAllocPtr alloc);
void MatchFinder_Free(CMatchFinder *p, ISzAllocPtr alloc);
void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, size_t numItems);
void MatchFinder_ReduceOffsets(CMatchFinder *p, UInt32 subValue);
UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *buffer, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
UInt32 *distances, UInt32 maxLen);
/*
Conditions:
Mf_GetNumAvailableBytes_Func must be called before each Mf_GetMatchLen_Func.
Mf_GetPointerToCurrentPos_Func's result must be used only before any other function
*/
typedef void (*Mf_Init_Func)(void *object);
typedef UInt32 (*Mf_GetNumAvailableBytes_Func)(void *object);
typedef const Byte * (*Mf_GetPointerToCurrentPos_Func)(void *object);
typedef UInt32 (*Mf_GetMatches_Func)(void *object, UInt32 *distances);
typedef void (*Mf_Skip_Func)(void *object, UInt32);
typedef struct _IMatchFinder
{
Mf_Init_Func Init;
Mf_GetNumAvailableBytes_Func GetNumAvailableBytes;
Mf_GetPointerToCurrentPos_Func GetPointerToCurrentPos;
Mf_GetMatches_Func GetMatches;
Mf_Skip_Func Skip;
} IMatchFinder;
void MatchFinder_CreateVTable(CMatchFinder *p, IMatchFinder *vTable);
void MatchFinder_Init_LowHash(CMatchFinder *p);
void MatchFinder_Init_HighHash(CMatchFinder *p);
void MatchFinder_Init_3(CMatchFinder *p, int readData);
void MatchFinder_Init(CMatchFinder *p);
UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances);
UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances);
void Bt3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num);
void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num);
EXTERN_C_END
#endif
+853
View File
@@ -0,0 +1,853 @@
/* LzFindMt.c -- multithreaded Match finder for LZ algorithms
2018-12-29 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "LzHash.h"
#include "LzFindMt.h"
static void MtSync_Construct(CMtSync *p)
{
p->wasCreated = False;
p->csWasInitialized = False;
p->csWasEntered = False;
Thread_Construct(&p->thread);
Event_Construct(&p->canStart);
Event_Construct(&p->wasStarted);
Event_Construct(&p->wasStopped);
Semaphore_Construct(&p->freeSemaphore);
Semaphore_Construct(&p->filledSemaphore);
}
static void MtSync_GetNextBlock(CMtSync *p)
{
if (p->needStart)
{
p->numProcessedBlocks = 1;
p->needStart = False;
p->stopWriting = False;
p->exit = False;
Event_Reset(&p->wasStarted);
Event_Reset(&p->wasStopped);
Event_Set(&p->canStart);
Event_Wait(&p->wasStarted);
// if (mt) MatchFinder_Init_LowHash(mt->MatchFinder);
}
else
{
CriticalSection_Leave(&p->cs);
p->csWasEntered = False;
p->numProcessedBlocks++;
Semaphore_Release1(&p->freeSemaphore);
}
Semaphore_Wait(&p->filledSemaphore);
CriticalSection_Enter(&p->cs);
p->csWasEntered = True;
}
/* MtSync_StopWriting must be called if Writing was started */
static void MtSync_StopWriting(CMtSync *p)
{
UInt32 myNumBlocks = p->numProcessedBlocks;
if (!Thread_WasCreated(&p->thread) || p->needStart)
return;
p->stopWriting = True;
if (p->csWasEntered)
{
CriticalSection_Leave(&p->cs);
p->csWasEntered = False;
}
Semaphore_Release1(&p->freeSemaphore);
Event_Wait(&p->wasStopped);
while (myNumBlocks++ != p->numProcessedBlocks)
{
Semaphore_Wait(&p->filledSemaphore);
Semaphore_Release1(&p->freeSemaphore);
}
p->needStart = True;
}
static void MtSync_Destruct(CMtSync *p)
{
if (Thread_WasCreated(&p->thread))
{
MtSync_StopWriting(p);
p->exit = True;
if (p->needStart)
Event_Set(&p->canStart);
Thread_Wait(&p->thread);
Thread_Close(&p->thread);
}
if (p->csWasInitialized)
{
CriticalSection_Delete(&p->cs);
p->csWasInitialized = False;
}
Event_Close(&p->canStart);
Event_Close(&p->wasStarted);
Event_Close(&p->wasStopped);
Semaphore_Close(&p->freeSemaphore);
Semaphore_Close(&p->filledSemaphore);
p->wasCreated = False;
}
#define RINOK_THREAD(x) { if ((x) != 0) return SZ_ERROR_THREAD; }
static SRes MtSync_Create2(CMtSync *p, THREAD_FUNC_TYPE startAddress, void *obj, UInt32 numBlocks)
{
if (p->wasCreated)
return SZ_OK;
RINOK_THREAD(CriticalSection_Init(&p->cs));
p->csWasInitialized = True;
RINOK_THREAD(AutoResetEvent_CreateNotSignaled(&p->canStart));
RINOK_THREAD(AutoResetEvent_CreateNotSignaled(&p->wasStarted));
RINOK_THREAD(AutoResetEvent_CreateNotSignaled(&p->wasStopped));
RINOK_THREAD(Semaphore_Create(&p->freeSemaphore, numBlocks, numBlocks));
RINOK_THREAD(Semaphore_Create(&p->filledSemaphore, 0, numBlocks));
p->needStart = True;
RINOK_THREAD(Thread_Create(&p->thread, startAddress, obj));
p->wasCreated = True;
return SZ_OK;
}
static SRes MtSync_Create(CMtSync *p, THREAD_FUNC_TYPE startAddress, void *obj, UInt32 numBlocks)
{
SRes res = MtSync_Create2(p, startAddress, obj, numBlocks);
if (res != SZ_OK)
MtSync_Destruct(p);
return res;
}
void MtSync_Init(CMtSync *p) { p->needStart = True; }
#define kMtMaxValForNormalize 0xFFFFFFFF
#define DEF_GetHeads2(name, v, action) \
static void GetHeads ## name(const Byte *p, UInt32 pos, \
UInt32 *hash, UInt32 hashMask, UInt32 *heads, UInt32 numHeads, const UInt32 *crc) \
{ action; for (; numHeads != 0; numHeads--) { \
const UInt32 value = (v); p++; *heads++ = pos - hash[value]; hash[value] = pos++; } }
#define DEF_GetHeads(name, v) DEF_GetHeads2(name, v, ;)
DEF_GetHeads2(2, (p[0] | ((UInt32)p[1] << 8)), UNUSED_VAR(hashMask); UNUSED_VAR(crc); )
DEF_GetHeads(3, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8)) & hashMask)
DEF_GetHeads(4, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ (crc[p[3]] << 5)) & hashMask)
DEF_GetHeads(4b, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ ((UInt32)p[3] << 16)) & hashMask)
/* DEF_GetHeads(5, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ (crc[p[3]] << 5) ^ (crc[p[4]] << 3)) & hashMask) */
static void HashThreadFunc(CMatchFinderMt *mt)
{
CMtSync *p = &mt->hashSync;
for (;;)
{
UInt32 numProcessedBlocks = 0;
Event_Wait(&p->canStart);
Event_Set(&p->wasStarted);
MatchFinder_Init_HighHash(mt->MatchFinder);
for (;;)
{
if (p->exit)
return;
if (p->stopWriting)
{
p->numProcessedBlocks = numProcessedBlocks;
Event_Set(&p->wasStopped);
break;
}
{
CMatchFinder *mf = mt->MatchFinder;
if (MatchFinder_NeedMove(mf))
{
CriticalSection_Enter(&mt->btSync.cs);
CriticalSection_Enter(&mt->hashSync.cs);
{
const Byte *beforePtr = Inline_MatchFinder_GetPointerToCurrentPos(mf);
ptrdiff_t offset;
MatchFinder_MoveBlock(mf);
offset = beforePtr - Inline_MatchFinder_GetPointerToCurrentPos(mf);
mt->pointerToCurPos -= offset;
mt->buffer -= offset;
}
CriticalSection_Leave(&mt->btSync.cs);
CriticalSection_Leave(&mt->hashSync.cs);
continue;
}
Semaphore_Wait(&p->freeSemaphore);
MatchFinder_ReadIfRequired(mf);
if (mf->pos > (kMtMaxValForNormalize - kMtHashBlockSize))
{
UInt32 subValue = (mf->pos - mf->historySize - 1);
MatchFinder_ReduceOffsets(mf, subValue);
MatchFinder_Normalize3(subValue, mf->hash + mf->fixedHashSize, (size_t)mf->hashMask + 1);
}
{
UInt32 *heads = mt->hashBuf + ((numProcessedBlocks++) & kMtHashNumBlocksMask) * kMtHashBlockSize;
UInt32 num = mf->streamPos - mf->pos;
heads[0] = 2;
heads[1] = num;
if (num >= mf->numHashBytes)
{
num = num - mf->numHashBytes + 1;
if (num > kMtHashBlockSize - 2)
num = kMtHashBlockSize - 2;
mt->GetHeadsFunc(mf->buffer, mf->pos, mf->hash + mf->fixedHashSize, mf->hashMask, heads + 2, num, mf->crc);
heads[0] = 2 + num;
}
mf->pos += num;
mf->buffer += num;
}
}
Semaphore_Release1(&p->filledSemaphore);
}
}
}
static void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
{
MtSync_GetNextBlock(&p->hashSync);
p->hashBufPosLimit = p->hashBufPos = ((p->hashSync.numProcessedBlocks - 1) & kMtHashNumBlocksMask) * kMtHashBlockSize;
p->hashBufPosLimit += p->hashBuf[p->hashBufPos++];
p->hashNumAvail = p->hashBuf[p->hashBufPos++];
}
#define kEmptyHashValue 0
#define MFMT_GM_INLINE
#ifdef MFMT_GM_INLINE
/*
we use size_t for _cyclicBufferPos instead of UInt32
to eliminate "movsx" BUG in old MSVC x64 compiler.
*/
MY_NO_INLINE
static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32 *limit, UInt32 size, UInt32 *posRes)
{
do
{
UInt32 *_distances = ++distances;
UInt32 delta = *hash++;
CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
unsigned len0 = 0, len1 = 0;
UInt32 cutValue = _cutValue;
unsigned maxLen = (unsigned)_maxLen;
/*
if (size > 1)
{
UInt32 delta = *hash;
if (delta < _cyclicBufferSize)
{
UInt32 cyc1 = _cyclicBufferPos + 1;
CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ? _cyclicBufferSize : 0)) << 1);
Byte b = *(cur + 1 - delta);
_distances[0] = pair[0];
_distances[1] = b;
}
}
*/
if (cutValue == 0 || delta >= _cyclicBufferSize)
{
*ptr0 = *ptr1 = kEmptyHashValue;
}
else
for(;;)
{
{
CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
unsigned len = (len0 < len1 ? len0 : len1);
UInt32 pair0 = *pair;
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
while (++len != lenLimit)
if (pb[len] != cur[len])
break;
if (maxLen < len)
{
maxLen = len;
*distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
UInt32 pair1 = pair[1];
*ptr1 = pair0;
*ptr0 = pair1;
break;
}
}
}
{
UInt32 curMatch = pos - delta;
// delta = pos - *pair;
// delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >> 31];
if (pb[len] < cur[len])
{
delta = pos - pair[1];
*ptr1 = curMatch;
ptr1 = pair + 1;
len1 = len;
}
else
{
delta = pos - *pair;
*ptr0 = curMatch;
ptr0 = pair;
len0 = len;
}
}
}
if (--cutValue == 0 || delta >= _cyclicBufferSize)
{
*ptr0 = *ptr1 = kEmptyHashValue;
break;
}
}
pos++;
_cyclicBufferPos++;
cur++;
{
UInt32 num = (UInt32)(distances - _distances);
_distances[-1] = num;
}
}
while (distances < limit && --size != 0);
*posRes = pos;
return distances;
}
#endif
static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
{
UInt32 numProcessed = 0;
UInt32 curPos = 2;
UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
distances[1] = p->hashNumAvail;
while (curPos < limit)
{
if (p->hashBufPos == p->hashBufPosLimit)
{
MatchFinderMt_GetNextBlock_Hash(p);
distances[1] = numProcessed + p->hashNumAvail;
if (p->hashNumAvail >= p->numHashBytes)
continue;
distances[0] = curPos + p->hashNumAvail;
distances += curPos;
for (; p->hashNumAvail != 0; p->hashNumAvail--)
*distances++ = 0;
return;
}
{
UInt32 size = p->hashBufPosLimit - p->hashBufPos;
UInt32 lenLimit = p->matchMaxLen;
UInt32 pos = p->pos;
UInt32 cyclicBufferPos = p->cyclicBufferPos;
if (lenLimit >= p->hashNumAvail)
lenLimit = p->hashNumAvail;
{
UInt32 size2 = p->hashNumAvail - lenLimit + 1;
if (size2 < size)
size = size2;
size2 = p->cyclicBufferSize - cyclicBufferPos;
if (size2 < size)
size = size2;
}
#ifndef MFMT_GM_INLINE
while (curPos < limit && size-- != 0)
{
UInt32 *startDistances = distances + curPos;
UInt32 num = (UInt32)(GetMatchesSpec1(lenLimit, pos - p->hashBuf[p->hashBufPos++],
pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
startDistances + 1, p->numHashBytes - 1) - startDistances);
*startDistances = num - 1;
curPos += num;
cyclicBufferPos++;
pos++;
p->buffer++;
}
#else
{
UInt32 posRes;
curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos,
distances + limit,
size, &posRes) - distances);
p->hashBufPos += posRes - pos;
cyclicBufferPos += posRes - pos;
p->buffer += posRes - pos;
pos = posRes;
}
#endif
numProcessed += pos - p->pos;
p->hashNumAvail -= pos - p->pos;
p->pos = pos;
if (cyclicBufferPos == p->cyclicBufferSize)
cyclicBufferPos = 0;
p->cyclicBufferPos = cyclicBufferPos;
}
}
distances[0] = curPos;
}
static void BtFillBlock(CMatchFinderMt *p, UInt32 globalBlockIndex)
{
CMtSync *sync = &p->hashSync;
if (!sync->needStart)
{
CriticalSection_Enter(&sync->cs);
sync->csWasEntered = True;
}
BtGetMatches(p, p->btBuf + (globalBlockIndex & kMtBtNumBlocksMask) * kMtBtBlockSize);
if (p->pos > kMtMaxValForNormalize - kMtBtBlockSize)
{
UInt32 subValue = p->pos - p->cyclicBufferSize;
MatchFinder_Normalize3(subValue, p->son, (size_t)p->cyclicBufferSize * 2);
p->pos -= subValue;
}
if (!sync->needStart)
{
CriticalSection_Leave(&sync->cs);
sync->csWasEntered = False;
}
}
void BtThreadFunc(CMatchFinderMt *mt)
{
CMtSync *p = &mt->btSync;
for (;;)
{
UInt32 blockIndex = 0;
Event_Wait(&p->canStart);
Event_Set(&p->wasStarted);
for (;;)
{
if (p->exit)
return;
if (p->stopWriting)
{
p->numProcessedBlocks = blockIndex;
MtSync_StopWriting(&mt->hashSync);
Event_Set(&p->wasStopped);
break;
}
Semaphore_Wait(&p->freeSemaphore);
BtFillBlock(mt, blockIndex++);
Semaphore_Release1(&p->filledSemaphore);
}
}
}
void MatchFinderMt_Construct(CMatchFinderMt *p)
{
p->hashBuf = NULL;
MtSync_Construct(&p->hashSync);
MtSync_Construct(&p->btSync);
}
static void MatchFinderMt_FreeMem(CMatchFinderMt *p, ISzAllocPtr alloc)
{
ISzAlloc_Free(alloc, p->hashBuf);
p->hashBuf = NULL;
}
void MatchFinderMt_Destruct(CMatchFinderMt *p, ISzAllocPtr alloc)
{
MtSync_Destruct(&p->hashSync);
MtSync_Destruct(&p->btSync);
MatchFinderMt_FreeMem(p, alloc);
}
#define kHashBufferSize (kMtHashBlockSize * kMtHashNumBlocks)
#define kBtBufferSize (kMtBtBlockSize * kMtBtNumBlocks)
static THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE HashThreadFunc2(void *p) { HashThreadFunc((CMatchFinderMt *)p); return 0; }
static THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE BtThreadFunc2(void *p)
{
Byte allocaDummy[0x180];
unsigned i = 0;
for (i = 0; i < 16; i++)
allocaDummy[i] = (Byte)0;
if (allocaDummy[0] == 0)
BtThreadFunc((CMatchFinderMt *)p);
return 0;
}
SRes MatchFinderMt_Create(CMatchFinderMt *p, UInt32 historySize, UInt32 keepAddBufferBefore,
UInt32 matchMaxLen, UInt32 keepAddBufferAfter, ISzAllocPtr alloc)
{
CMatchFinder *mf = p->MatchFinder;
p->historySize = historySize;
if (kMtBtBlockSize <= matchMaxLen * 4)
return SZ_ERROR_PARAM;
if (!p->hashBuf)
{
p->hashBuf = (UInt32 *)ISzAlloc_Alloc(alloc, (kHashBufferSize + kBtBufferSize) * sizeof(UInt32));
if (!p->hashBuf)
return SZ_ERROR_MEM;
p->btBuf = p->hashBuf + kHashBufferSize;
}
keepAddBufferBefore += (kHashBufferSize + kBtBufferSize);
keepAddBufferAfter += kMtHashBlockSize;
if (!MatchFinder_Create(mf, historySize, keepAddBufferBefore, matchMaxLen, keepAddBufferAfter, alloc))
return SZ_ERROR_MEM;
RINOK(MtSync_Create(&p->hashSync, HashThreadFunc2, p, kMtHashNumBlocks));
RINOK(MtSync_Create(&p->btSync, BtThreadFunc2, p, kMtBtNumBlocks));
return SZ_OK;
}
/* Call it after ReleaseStream / SetStream */
static void MatchFinderMt_Init(CMatchFinderMt *p)
{
CMatchFinder *mf = p->MatchFinder;
p->btBufPos =
p->btBufPosLimit = 0;
p->hashBufPos =
p->hashBufPosLimit = 0;
/* Init without data reading. We don't want to read data in this thread */
MatchFinder_Init_3(mf, False);
MatchFinder_Init_LowHash(mf);
p->pointerToCurPos = Inline_MatchFinder_GetPointerToCurrentPos(mf);
p->btNumAvailBytes = 0;
p->lzPos = p->historySize + 1;
p->hash = mf->hash;
p->fixedHashSize = mf->fixedHashSize;
p->crc = mf->crc;
p->son = mf->son;
p->matchMaxLen = mf->matchMaxLen;
p->numHashBytes = mf->numHashBytes;
p->pos = mf->pos;
p->buffer = mf->buffer;
p->cyclicBufferPos = mf->cyclicBufferPos;
p->cyclicBufferSize = mf->cyclicBufferSize;
p->cutValue = mf->cutValue;
}
/* ReleaseStream is required to finish multithreading */
void MatchFinderMt_ReleaseStream(CMatchFinderMt *p)
{
MtSync_StopWriting(&p->btSync);
/* p->MatchFinder->ReleaseStream(); */
}
static void MatchFinderMt_Normalize(CMatchFinderMt *p)
{
MatchFinder_Normalize3(p->lzPos - p->historySize - 1, p->hash, p->fixedHashSize);
p->lzPos = p->historySize + 1;
}
static void MatchFinderMt_GetNextBlock_Bt(CMatchFinderMt *p)
{
UInt32 blockIndex;
MtSync_GetNextBlock(&p->btSync);
blockIndex = ((p->btSync.numProcessedBlocks - 1) & kMtBtNumBlocksMask);
p->btBufPosLimit = p->btBufPos = blockIndex * kMtBtBlockSize;
p->btBufPosLimit += p->btBuf[p->btBufPos++];
p->btNumAvailBytes = p->btBuf[p->btBufPos++];
if (p->lzPos >= kMtMaxValForNormalize - kMtBtBlockSize)
MatchFinderMt_Normalize(p);
}
static const Byte * MatchFinderMt_GetPointerToCurrentPos(CMatchFinderMt *p)
{
return p->pointerToCurPos;
}
#define GET_NEXT_BLOCK_IF_REQUIRED if (p->btBufPos == p->btBufPosLimit) MatchFinderMt_GetNextBlock_Bt(p);
static UInt32 MatchFinderMt_GetNumAvailableBytes(CMatchFinderMt *p)
{
GET_NEXT_BLOCK_IF_REQUIRED;
return p->btNumAvailBytes;
}
static UInt32 * MixMatches2(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *distances)
{
UInt32 h2, curMatch2;
UInt32 *hash = p->hash;
const Byte *cur = p->pointerToCurPos;
UInt32 lzPos = p->lzPos;
MT_HASH2_CALC
curMatch2 = hash[h2];
hash[h2] = lzPos;
if (curMatch2 >= matchMinPos)
if (cur[(ptrdiff_t)curMatch2 - lzPos] == cur[0])
{
*distances++ = 2;
*distances++ = lzPos - curMatch2 - 1;
}
return distances;
}
static UInt32 * MixMatches3(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *distances)
{
UInt32 h2, h3, curMatch2, curMatch3;
UInt32 *hash = p->hash;
const Byte *cur = p->pointerToCurPos;
UInt32 lzPos = p->lzPos;
MT_HASH3_CALC
curMatch2 = hash[ h2];
curMatch3 = (hash + kFix3HashSize)[h3];
hash[ h2] = lzPos;
(hash + kFix3HashSize)[h3] = lzPos;
if (curMatch2 >= matchMinPos && cur[(ptrdiff_t)curMatch2 - lzPos] == cur[0])
{
distances[1] = lzPos - curMatch2 - 1;
if (cur[(ptrdiff_t)curMatch2 - lzPos + 2] == cur[2])
{
distances[0] = 3;
return distances + 2;
}
distances[0] = 2;
distances += 2;
}
if (curMatch3 >= matchMinPos && cur[(ptrdiff_t)curMatch3 - lzPos] == cur[0])
{
*distances++ = 3;
*distances++ = lzPos - curMatch3 - 1;
}
return distances;
}
/*
static UInt32 *MixMatches4(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *distances)
{
UInt32 h2, h3, h4, curMatch2, curMatch3, curMatch4;
UInt32 *hash = p->hash;
const Byte *cur = p->pointerToCurPos;
UInt32 lzPos = p->lzPos;
MT_HASH4_CALC
curMatch2 = hash[ h2];
curMatch3 = (hash + kFix3HashSize)[h3];
curMatch4 = (hash + kFix4HashSize)[h4];
hash[ h2] = lzPos;
(hash + kFix3HashSize)[h3] = lzPos;
(hash + kFix4HashSize)[h4] = lzPos;
if (curMatch2 >= matchMinPos && cur[(ptrdiff_t)curMatch2 - lzPos] == cur[0])
{
distances[1] = lzPos - curMatch2 - 1;
if (cur[(ptrdiff_t)curMatch2 - lzPos + 2] == cur[2])
{
distances[0] = (cur[(ptrdiff_t)curMatch2 - lzPos + 3] == cur[3]) ? 4 : 3;
return distances + 2;
}
distances[0] = 2;
distances += 2;
}
if (curMatch3 >= matchMinPos && cur[(ptrdiff_t)curMatch3 - lzPos] == cur[0])
{
distances[1] = lzPos - curMatch3 - 1;
if (cur[(ptrdiff_t)curMatch3 - lzPos + 3] == cur[3])
{
distances[0] = 4;
return distances + 2;
}
distances[0] = 3;
distances += 2;
}
if (curMatch4 >= matchMinPos)
if (
cur[(ptrdiff_t)curMatch4 - lzPos] == cur[0] &&
cur[(ptrdiff_t)curMatch4 - lzPos + 3] == cur[3]
)
{
*distances++ = 4;
*distances++ = lzPos - curMatch4 - 1;
}
return distances;
}
*/
#define INCREASE_LZ_POS p->lzPos++; p->pointerToCurPos++;
static UInt32 MatchFinderMt2_GetMatches(CMatchFinderMt *p, UInt32 *distances)
{
const UInt32 *btBuf = p->btBuf + p->btBufPos;
UInt32 len = *btBuf++;
p->btBufPos += 1 + len;
p->btNumAvailBytes--;
{
UInt32 i;
for (i = 0; i < len; i += 2)
{
UInt32 v0 = btBuf[0];
UInt32 v1 = btBuf[1];
btBuf += 2;
distances[0] = v0;
distances[1] = v1;
distances += 2;
}
}
INCREASE_LZ_POS
return len;
}
static UInt32 MatchFinderMt_GetMatches(CMatchFinderMt *p, UInt32 *distances)
{
const UInt32 *btBuf = p->btBuf + p->btBufPos;
UInt32 len = *btBuf++;
p->btBufPos += 1 + len;
if (len == 0)
{
/* change for bt5 ! */
if (p->btNumAvailBytes-- >= 4)
len = (UInt32)(p->MixMatchesFunc(p, p->lzPos - p->historySize, distances) - (distances));
}
else
{
/* Condition: there are matches in btBuf with length < p->numHashBytes */
UInt32 *distances2;
p->btNumAvailBytes--;
distances2 = p->MixMatchesFunc(p, p->lzPos - btBuf[1], distances);
do
{
UInt32 v0 = btBuf[0];
UInt32 v1 = btBuf[1];
btBuf += 2;
distances2[0] = v0;
distances2[1] = v1;
distances2 += 2;
}
while ((len -= 2) != 0);
len = (UInt32)(distances2 - (distances));
}
INCREASE_LZ_POS
return len;
}
#define SKIP_HEADER2_MT do { GET_NEXT_BLOCK_IF_REQUIRED
#define SKIP_HEADER_MT(n) SKIP_HEADER2_MT if (p->btNumAvailBytes-- >= (n)) { const Byte *cur = p->pointerToCurPos; UInt32 *hash = p->hash;
#define SKIP_FOOTER_MT } INCREASE_LZ_POS p->btBufPos += p->btBuf[p->btBufPos] + 1; } while (--num != 0);
static void MatchFinderMt0_Skip(CMatchFinderMt *p, UInt32 num)
{
SKIP_HEADER2_MT { p->btNumAvailBytes--;
SKIP_FOOTER_MT
}
static void MatchFinderMt2_Skip(CMatchFinderMt *p, UInt32 num)
{
SKIP_HEADER_MT(2)
UInt32 h2;
MT_HASH2_CALC
hash[h2] = p->lzPos;
SKIP_FOOTER_MT
}
static void MatchFinderMt3_Skip(CMatchFinderMt *p, UInt32 num)
{
SKIP_HEADER_MT(3)
UInt32 h2, h3;
MT_HASH3_CALC
(hash + kFix3HashSize)[h3] =
hash[ h2] =
p->lzPos;
SKIP_FOOTER_MT
}
/*
static void MatchFinderMt4_Skip(CMatchFinderMt *p, UInt32 num)
{
SKIP_HEADER_MT(4)
UInt32 h2, h3, h4;
MT_HASH4_CALC
(hash + kFix4HashSize)[h4] =
(hash + kFix3HashSize)[h3] =
hash[ h2] =
p->lzPos;
SKIP_FOOTER_MT
}
*/
void MatchFinderMt_CreateVTable(CMatchFinderMt *p, IMatchFinder *vTable)
{
vTable->Init = (Mf_Init_Func)MatchFinderMt_Init;
vTable->GetNumAvailableBytes = (Mf_GetNumAvailableBytes_Func)MatchFinderMt_GetNumAvailableBytes;
vTable->GetPointerToCurrentPos = (Mf_GetPointerToCurrentPos_Func)MatchFinderMt_GetPointerToCurrentPos;
vTable->GetMatches = (Mf_GetMatches_Func)MatchFinderMt_GetMatches;
switch (p->MatchFinder->numHashBytes)
{
case 2:
p->GetHeadsFunc = GetHeads2;
p->MixMatchesFunc = (Mf_Mix_Matches)NULL;
vTable->Skip = (Mf_Skip_Func)MatchFinderMt0_Skip;
vTable->GetMatches = (Mf_GetMatches_Func)MatchFinderMt2_GetMatches;
break;
case 3:
p->GetHeadsFunc = GetHeads3;
p->MixMatchesFunc = (Mf_Mix_Matches)MixMatches2;
vTable->Skip = (Mf_Skip_Func)MatchFinderMt2_Skip;
break;
default:
/* case 4: */
p->GetHeadsFunc = p->MatchFinder->bigHash ? GetHeads4b : GetHeads4;
p->MixMatchesFunc = (Mf_Mix_Matches)MixMatches3;
vTable->Skip = (Mf_Skip_Func)MatchFinderMt3_Skip;
break;
/*
default:
p->GetHeadsFunc = GetHeads5;
p->MixMatchesFunc = (Mf_Mix_Matches)MixMatches4;
vTable->Skip = (Mf_Skip_Func)MatchFinderMt4_Skip;
break;
*/
}
}
+101
View File
@@ -0,0 +1,101 @@
/* LzFindMt.h -- multithreaded Match finder for LZ algorithms
2018-07-04 : Igor Pavlov : Public domain */
#ifndef __LZ_FIND_MT_H
#define __LZ_FIND_MT_H
#include "LzFind.h"
#include "Threads.h"
EXTERN_C_BEGIN
#define kMtHashBlockSize (1 << 13)
#define kMtHashNumBlocks (1 << 3)
#define kMtHashNumBlocksMask (kMtHashNumBlocks - 1)
#define kMtBtBlockSize (1 << 14)
#define kMtBtNumBlocks (1 << 6)
#define kMtBtNumBlocksMask (kMtBtNumBlocks - 1)
typedef struct _CMtSync
{
BoolInt wasCreated;
BoolInt needStart;
BoolInt exit;
BoolInt stopWriting;
CThread thread;
CAutoResetEvent canStart;
CAutoResetEvent wasStarted;
CAutoResetEvent wasStopped;
CSemaphore freeSemaphore;
CSemaphore filledSemaphore;
BoolInt csWasInitialized;
BoolInt csWasEntered;
CCriticalSection cs;
UInt32 numProcessedBlocks;
} CMtSync;
typedef UInt32 * (*Mf_Mix_Matches)(void *p, UInt32 matchMinPos, UInt32 *distances);
/* kMtCacheLineDummy must be >= size_of_CPU_cache_line */
#define kMtCacheLineDummy 128
typedef void (*Mf_GetHeads)(const Byte *buffer, UInt32 pos,
UInt32 *hash, UInt32 hashMask, UInt32 *heads, UInt32 numHeads, const UInt32 *crc);
typedef struct _CMatchFinderMt
{
/* LZ */
const Byte *pointerToCurPos;
UInt32 *btBuf;
UInt32 btBufPos;
UInt32 btBufPosLimit;
UInt32 lzPos;
UInt32 btNumAvailBytes;
UInt32 *hash;
UInt32 fixedHashSize;
UInt32 historySize;
const UInt32 *crc;
Mf_Mix_Matches MixMatchesFunc;
/* LZ + BT */
CMtSync btSync;
Byte btDummy[kMtCacheLineDummy];
/* BT */
UInt32 *hashBuf;
UInt32 hashBufPos;
UInt32 hashBufPosLimit;
UInt32 hashNumAvail;
CLzRef *son;
UInt32 matchMaxLen;
UInt32 numHashBytes;
UInt32 pos;
const Byte *buffer;
UInt32 cyclicBufferPos;
UInt32 cyclicBufferSize; /* it must be historySize + 1 */
UInt32 cutValue;
/* BT + Hash */
CMtSync hashSync;
/* Byte hashDummy[kMtCacheLineDummy]; */
/* Hash */
Mf_GetHeads GetHeadsFunc;
CMatchFinder *MatchFinder;
} CMatchFinderMt;
void MatchFinderMt_Construct(CMatchFinderMt *p);
void MatchFinderMt_Destruct(CMatchFinderMt *p, ISzAllocPtr alloc);
SRes MatchFinderMt_Create(CMatchFinderMt *p, UInt32 historySize, UInt32 keepAddBufferBefore,
UInt32 matchMaxLen, UInt32 keepAddBufferAfter, ISzAllocPtr alloc);
void MatchFinderMt_CreateVTable(CMatchFinderMt *p, IMatchFinder *vTable);
void MatchFinderMt_ReleaseStream(CMatchFinderMt *p);
EXTERN_C_END
#endif
+57
View File
@@ -0,0 +1,57 @@
/* LzHash.h -- HASH functions for LZ algorithms
2015-04-12 : Igor Pavlov : Public domain */
#ifndef __LZ_HASH_H
#define __LZ_HASH_H
#define kHash2Size (1 << 10)
#define kHash3Size (1 << 16)
#define kHash4Size (1 << 20)
#define kFix3HashSize (kHash2Size)
#define kFix4HashSize (kHash2Size + kHash3Size)
#define kFix5HashSize (kHash2Size + kHash3Size + kHash4Size)
#define HASH2_CALC hv = cur[0] | ((UInt32)cur[1] << 8);
#define HASH3_CALC { \
UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
h2 = temp & (kHash2Size - 1); \
hv = (temp ^ ((UInt32)cur[2] << 8)) & p->hashMask; }
#define HASH4_CALC { \
UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
h2 = temp & (kHash2Size - 1); \
temp ^= ((UInt32)cur[2] << 8); \
h3 = temp & (kHash3Size - 1); \
hv = (temp ^ (p->crc[cur[3]] << 5)) & p->hashMask; }
#define HASH5_CALC { \
UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
h2 = temp & (kHash2Size - 1); \
temp ^= ((UInt32)cur[2] << 8); \
h3 = temp & (kHash3Size - 1); \
temp ^= (p->crc[cur[3]] << 5); \
h4 = temp & (kHash4Size - 1); \
hv = (temp ^ (p->crc[cur[4]] << 3)) & p->hashMask; }
/* #define HASH_ZIP_CALC hv = ((cur[0] | ((UInt32)cur[1] << 8)) ^ p->crc[cur[2]]) & 0xFFFF; */
#define HASH_ZIP_CALC hv = ((cur[2] | ((UInt32)cur[0] << 8)) ^ p->crc[cur[1]]) & 0xFFFF;
#define MT_HASH2_CALC \
h2 = (p->crc[cur[0]] ^ cur[1]) & (kHash2Size - 1);
#define MT_HASH3_CALC { \
UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
h2 = temp & (kHash2Size - 1); \
h3 = (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); }
#define MT_HASH4_CALC { \
UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
h2 = temp & (kHash2Size - 1); \
temp ^= ((UInt32)cur[2] << 8); \
h3 = temp & (kHash3Size - 1); \
h4 = (temp ^ (p->crc[cur[3]] << 5)) & (kHash4Size - 1); }
#endif
+488
View File
@@ -0,0 +1,488 @@
/* Lzma2Dec.c -- LZMA2 Decoder
2019-02-02 : Igor Pavlov : Public domain */
/* #define SHOW_DEBUG_INFO */
#include "Precomp.h"
#ifdef SHOW_DEBUG_INFO
#include <stdio.h>
#endif
#include <string.h>
#include "Lzma2Dec.h"
/*
00000000 - End of data
00000001 U U - Uncompressed, reset dic, need reset state and set new prop
00000010 U U - Uncompressed, no reset
100uuuuu U U P P - LZMA, no reset
101uuuuu U U P P - LZMA, reset state
110uuuuu U U P P S - LZMA, reset state + set new prop
111uuuuu U U P P S - LZMA, reset state + set new prop, reset dic
u, U - Unpack Size
P - Pack Size
S - Props
*/
#define LZMA2_CONTROL_COPY_RESET_DIC 1
#define LZMA2_IS_UNCOMPRESSED_STATE(p) (((p)->control & (1 << 7)) == 0)
#define LZMA2_LCLP_MAX 4
#define LZMA2_DIC_SIZE_FROM_PROP(p) (((UInt32)2 | ((p) & 1)) << ((p) / 2 + 11))
#ifdef SHOW_DEBUG_INFO
#define PRF(x) x
#else
#define PRF(x)
#endif
typedef enum
{
LZMA2_STATE_CONTROL,
LZMA2_STATE_UNPACK0,
LZMA2_STATE_UNPACK1,
LZMA2_STATE_PACK0,
LZMA2_STATE_PACK1,
LZMA2_STATE_PROP,
LZMA2_STATE_DATA,
LZMA2_STATE_DATA_CONT,
LZMA2_STATE_FINISHED,
LZMA2_STATE_ERROR
} ELzma2State;
static SRes Lzma2Dec_GetOldProps(Byte prop, Byte *props)
{
UInt32 dicSize;
if (prop > 40)
return SZ_ERROR_UNSUPPORTED;
dicSize = (prop == 40) ? 0xFFFFFFFF : LZMA2_DIC_SIZE_FROM_PROP(prop);
props[0] = (Byte)LZMA2_LCLP_MAX;
props[1] = (Byte)(dicSize);
props[2] = (Byte)(dicSize >> 8);
props[3] = (Byte)(dicSize >> 16);
props[4] = (Byte)(dicSize >> 24);
return SZ_OK;
}
SRes Lzma2Dec_AllocateProbs(CLzma2Dec *p, Byte prop, ISzAllocPtr alloc)
{
Byte props[LZMA_PROPS_SIZE];
RINOK(Lzma2Dec_GetOldProps(prop, props));
return LzmaDec_AllocateProbs(&p->decoder, props, LZMA_PROPS_SIZE, alloc);
}
SRes Lzma2Dec_Allocate(CLzma2Dec *p, Byte prop, ISzAllocPtr alloc)
{
Byte props[LZMA_PROPS_SIZE];
RINOK(Lzma2Dec_GetOldProps(prop, props));
return LzmaDec_Allocate(&p->decoder, props, LZMA_PROPS_SIZE, alloc);
}
void Lzma2Dec_Init(CLzma2Dec *p)
{
p->state = LZMA2_STATE_CONTROL;
p->needInitLevel = 0xE0;
p->isExtraMode = False;
p->unpackSize = 0;
// p->decoder.dicPos = 0; // we can use it instead of full init
LzmaDec_Init(&p->decoder);
}
static ELzma2State Lzma2Dec_UpdateState(CLzma2Dec *p, Byte b)
{
switch (p->state)
{
case LZMA2_STATE_CONTROL:
p->isExtraMode = False;
p->control = b;
PRF(printf("\n %8X", (unsigned)p->decoder.dicPos));
PRF(printf(" %02X", (unsigned)b));
if (b == 0)
return LZMA2_STATE_FINISHED;
if (LZMA2_IS_UNCOMPRESSED_STATE(p))
{
if (b == LZMA2_CONTROL_COPY_RESET_DIC)
p->needInitLevel = 0xC0;
else if (b > 2 || p->needInitLevel == 0xE0)
return LZMA2_STATE_ERROR;
}
else
{
if (b < p->needInitLevel)
return LZMA2_STATE_ERROR;
p->needInitLevel = 0;
p->unpackSize = (UInt32)(b & 0x1F) << 16;
}
return LZMA2_STATE_UNPACK0;
case LZMA2_STATE_UNPACK0:
p->unpackSize |= (UInt32)b << 8;
return LZMA2_STATE_UNPACK1;
case LZMA2_STATE_UNPACK1:
p->unpackSize |= (UInt32)b;
p->unpackSize++;
PRF(printf(" %7u", (unsigned)p->unpackSize));
return LZMA2_IS_UNCOMPRESSED_STATE(p) ? LZMA2_STATE_DATA : LZMA2_STATE_PACK0;
case LZMA2_STATE_PACK0:
p->packSize = (UInt32)b << 8;
return LZMA2_STATE_PACK1;
case LZMA2_STATE_PACK1:
p->packSize |= (UInt32)b;
p->packSize++;
// if (p->packSize < 5) return LZMA2_STATE_ERROR;
PRF(printf(" %5u", (unsigned)p->packSize));
return (p->control & 0x40) ? LZMA2_STATE_PROP : LZMA2_STATE_DATA;
case LZMA2_STATE_PROP:
{
unsigned lc, lp;
if (b >= (9 * 5 * 5))
return LZMA2_STATE_ERROR;
lc = b % 9;
b /= 9;
p->decoder.prop.pb = (Byte)(b / 5);
lp = b % 5;
if (lc + lp > LZMA2_LCLP_MAX)
return LZMA2_STATE_ERROR;
p->decoder.prop.lc = (Byte)lc;
p->decoder.prop.lp = (Byte)lp;
return LZMA2_STATE_DATA;
}
}
return LZMA2_STATE_ERROR;
}
static void LzmaDec_UpdateWithUncompressed(CLzmaDec *p, const Byte *src, SizeT size)
{
memcpy(p->dic + p->dicPos, src, size);
p->dicPos += size;
if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= size)
p->checkDicSize = p->prop.dicSize;
p->processedPos += (UInt32)size;
}
void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt initState);
SRes Lzma2Dec_DecodeToDic(CLzma2Dec *p, SizeT dicLimit,
const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status)
{
SizeT inSize = *srcLen;
*srcLen = 0;
*status = LZMA_STATUS_NOT_SPECIFIED;
while (p->state != LZMA2_STATE_ERROR)
{
SizeT dicPos;
if (p->state == LZMA2_STATE_FINISHED)
{
*status = LZMA_STATUS_FINISHED_WITH_MARK;
return SZ_OK;
}
dicPos = p->decoder.dicPos;
if (dicPos == dicLimit && finishMode == LZMA_FINISH_ANY)
{
*status = LZMA_STATUS_NOT_FINISHED;
return SZ_OK;
}
if (p->state != LZMA2_STATE_DATA && p->state != LZMA2_STATE_DATA_CONT)
{
if (*srcLen == inSize)
{
*status = LZMA_STATUS_NEEDS_MORE_INPUT;
return SZ_OK;
}
(*srcLen)++;
p->state = Lzma2Dec_UpdateState(p, *src++);
if (dicPos == dicLimit && p->state != LZMA2_STATE_FINISHED)
break;
continue;
}
{
SizeT inCur = inSize - *srcLen;
SizeT outCur = dicLimit - dicPos;
ELzmaFinishMode curFinishMode = LZMA_FINISH_ANY;
if (outCur >= p->unpackSize)
{
outCur = (SizeT)p->unpackSize;
curFinishMode = LZMA_FINISH_END;
}
if (LZMA2_IS_UNCOMPRESSED_STATE(p))
{
if (inCur == 0)
{
*status = LZMA_STATUS_NEEDS_MORE_INPUT;
return SZ_OK;
}
if (p->state == LZMA2_STATE_DATA)
{
BoolInt initDic = (p->control == LZMA2_CONTROL_COPY_RESET_DIC);
LzmaDec_InitDicAndState(&p->decoder, initDic, False);
}
if (inCur > outCur)
inCur = outCur;
if (inCur == 0)
break;
LzmaDec_UpdateWithUncompressed(&p->decoder, src, inCur);
src += inCur;
*srcLen += inCur;
p->unpackSize -= (UInt32)inCur;
p->state = (p->unpackSize == 0) ? LZMA2_STATE_CONTROL : LZMA2_STATE_DATA_CONT;
}
else
{
SRes res;
if (p->state == LZMA2_STATE_DATA)
{
BoolInt initDic = (p->control >= 0xE0);
BoolInt initState = (p->control >= 0xA0);
LzmaDec_InitDicAndState(&p->decoder, initDic, initState);
p->state = LZMA2_STATE_DATA_CONT;
}
if (inCur > p->packSize)
inCur = (SizeT)p->packSize;
res = LzmaDec_DecodeToDic(&p->decoder, dicPos + outCur, src, &inCur, curFinishMode, status);
src += inCur;
*srcLen += inCur;
p->packSize -= (UInt32)inCur;
outCur = p->decoder.dicPos - dicPos;
p->unpackSize -= (UInt32)outCur;
if (res != 0)
break;
if (*status == LZMA_STATUS_NEEDS_MORE_INPUT)
{
if (p->packSize == 0)
break;
return SZ_OK;
}
if (inCur == 0 && outCur == 0)
{
if (*status != LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK
|| p->unpackSize != 0
|| p->packSize != 0)
break;
p->state = LZMA2_STATE_CONTROL;
}
*status = LZMA_STATUS_NOT_SPECIFIED;
}
}
}
*status = LZMA_STATUS_NOT_SPECIFIED;
p->state = LZMA2_STATE_ERROR;
return SZ_ERROR_DATA;
}
ELzma2ParseStatus Lzma2Dec_Parse(CLzma2Dec *p,
SizeT outSize,
const Byte *src, SizeT *srcLen,
int checkFinishBlock)
{
SizeT inSize = *srcLen;
*srcLen = 0;
while (p->state != LZMA2_STATE_ERROR)
{
if (p->state == LZMA2_STATE_FINISHED)
return (ELzma2ParseStatus)LZMA_STATUS_FINISHED_WITH_MARK;
if (outSize == 0 && !checkFinishBlock)
return (ELzma2ParseStatus)LZMA_STATUS_NOT_FINISHED;
if (p->state != LZMA2_STATE_DATA && p->state != LZMA2_STATE_DATA_CONT)
{
if (*srcLen == inSize)
return (ELzma2ParseStatus)LZMA_STATUS_NEEDS_MORE_INPUT;
(*srcLen)++;
p->state = Lzma2Dec_UpdateState(p, *src++);
if (p->state == LZMA2_STATE_UNPACK0)
{
// if (p->decoder.dicPos != 0)
if (p->control == LZMA2_CONTROL_COPY_RESET_DIC || p->control >= 0xE0)
return LZMA2_PARSE_STATUS_NEW_BLOCK;
// if (outSize == 0) return LZMA_STATUS_NOT_FINISHED;
}
// The following code can be commented.
// It's not big problem, if we read additional input bytes.
// It will be stopped later in LZMA2_STATE_DATA / LZMA2_STATE_DATA_CONT state.
if (outSize == 0 && p->state != LZMA2_STATE_FINISHED)
{
// checkFinishBlock is true. So we expect that block must be finished,
// We can return LZMA_STATUS_NOT_SPECIFIED or LZMA_STATUS_NOT_FINISHED here
// break;
return (ELzma2ParseStatus)LZMA_STATUS_NOT_FINISHED;
}
if (p->state == LZMA2_STATE_DATA)
return LZMA2_PARSE_STATUS_NEW_CHUNK;
continue;
}
if (outSize == 0)
return (ELzma2ParseStatus)LZMA_STATUS_NOT_FINISHED;
{
SizeT inCur = inSize - *srcLen;
if (LZMA2_IS_UNCOMPRESSED_STATE(p))
{
if (inCur == 0)
return (ELzma2ParseStatus)LZMA_STATUS_NEEDS_MORE_INPUT;
if (inCur > p->unpackSize)
inCur = p->unpackSize;
if (inCur > outSize)
inCur = outSize;
p->decoder.dicPos += inCur;
src += inCur;
*srcLen += inCur;
outSize -= inCur;
p->unpackSize -= (UInt32)inCur;
p->state = (p->unpackSize == 0) ? LZMA2_STATE_CONTROL : LZMA2_STATE_DATA_CONT;
}
else
{
p->isExtraMode = True;
if (inCur == 0)
{
if (p->packSize != 0)
return (ELzma2ParseStatus)LZMA_STATUS_NEEDS_MORE_INPUT;
}
else if (p->state == LZMA2_STATE_DATA)
{
p->state = LZMA2_STATE_DATA_CONT;
if (*src != 0)
{
// first byte of lzma chunk must be Zero
*srcLen += 1;
p->packSize--;
break;
}
}
if (inCur > p->packSize)
inCur = (SizeT)p->packSize;
src += inCur;
*srcLen += inCur;
p->packSize -= (UInt32)inCur;
if (p->packSize == 0)
{
SizeT rem = outSize;
if (rem > p->unpackSize)
rem = p->unpackSize;
p->decoder.dicPos += rem;
p->unpackSize -= (UInt32)rem;
outSize -= rem;
if (p->unpackSize == 0)
p->state = LZMA2_STATE_CONTROL;
}
}
}
}
p->state = LZMA2_STATE_ERROR;
return (ELzma2ParseStatus)LZMA_STATUS_NOT_SPECIFIED;
}
SRes Lzma2Dec_DecodeToBuf(CLzma2Dec *p, Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status)
{
SizeT outSize = *destLen, inSize = *srcLen;
*srcLen = *destLen = 0;
for (;;)
{
SizeT inCur = inSize, outCur, dicPos;
ELzmaFinishMode curFinishMode;
SRes res;
if (p->decoder.dicPos == p->decoder.dicBufSize)
p->decoder.dicPos = 0;
dicPos = p->decoder.dicPos;
curFinishMode = LZMA_FINISH_ANY;
outCur = p->decoder.dicBufSize - dicPos;
if (outCur >= outSize)
{
outCur = outSize;
curFinishMode = finishMode;
}
res = Lzma2Dec_DecodeToDic(p, dicPos + outCur, src, &inCur, curFinishMode, status);
src += inCur;
inSize -= inCur;
*srcLen += inCur;
outCur = p->decoder.dicPos - dicPos;
memcpy(dest, p->decoder.dic + dicPos, outCur);
dest += outCur;
outSize -= outCur;
*destLen += outCur;
if (res != 0)
return res;
if (outCur == 0 || outSize == 0)
return SZ_OK;
}
}
SRes Lzma2Decode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
Byte prop, ELzmaFinishMode finishMode, ELzmaStatus *status, ISzAllocPtr alloc)
{
CLzma2Dec p;
SRes res;
SizeT outSize = *destLen, inSize = *srcLen;
*destLen = *srcLen = 0;
*status = LZMA_STATUS_NOT_SPECIFIED;
Lzma2Dec_Construct(&p);
RINOK(Lzma2Dec_AllocateProbs(&p, prop, alloc));
p.decoder.dic = dest;
p.decoder.dicBufSize = outSize;
Lzma2Dec_Init(&p);
*srcLen = inSize;
res = Lzma2Dec_DecodeToDic(&p, outSize, src, srcLen, finishMode, status);
*destLen = p.decoder.dicPos;
if (res == SZ_OK && *status == LZMA_STATUS_NEEDS_MORE_INPUT)
res = SZ_ERROR_INPUT_EOF;
Lzma2Dec_FreeProbs(&p, alloc);
return res;
}
+120
View File
@@ -0,0 +1,120 @@
/* Lzma2Dec.h -- LZMA2 Decoder
2018-02-19 : Igor Pavlov : Public domain */
#ifndef __LZMA2_DEC_H
#define __LZMA2_DEC_H
#include "LzmaDec.h"
EXTERN_C_BEGIN
/* ---------- State Interface ---------- */
typedef struct
{
unsigned state;
Byte control;
Byte needInitLevel;
Byte isExtraMode;
Byte _pad_;
UInt32 packSize;
UInt32 unpackSize;
CLzmaDec decoder;
} CLzma2Dec;
#define Lzma2Dec_Construct(p) LzmaDec_Construct(&(p)->decoder)
#define Lzma2Dec_FreeProbs(p, alloc) LzmaDec_FreeProbs(&(p)->decoder, alloc)
#define Lzma2Dec_Free(p, alloc) LzmaDec_Free(&(p)->decoder, alloc)
SRes Lzma2Dec_AllocateProbs(CLzma2Dec *p, Byte prop, ISzAllocPtr alloc);
SRes Lzma2Dec_Allocate(CLzma2Dec *p, Byte prop, ISzAllocPtr alloc);
void Lzma2Dec_Init(CLzma2Dec *p);
/*
finishMode:
It has meaning only if the decoding reaches output limit (*destLen or dicLimit).
LZMA_FINISH_ANY - use smallest number of input bytes
LZMA_FINISH_END - read EndOfStream marker after decoding
Returns:
SZ_OK
status:
LZMA_STATUS_FINISHED_WITH_MARK
LZMA_STATUS_NOT_FINISHED
LZMA_STATUS_NEEDS_MORE_INPUT
SZ_ERROR_DATA - Data error
*/
SRes Lzma2Dec_DecodeToDic(CLzma2Dec *p, SizeT dicLimit,
const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);
SRes Lzma2Dec_DecodeToBuf(CLzma2Dec *p, Byte *dest, SizeT *destLen,
const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);
/* ---------- LZMA2 block and chunk parsing ---------- */
/*
Lzma2Dec_Parse() parses compressed data stream up to next independent block or next chunk data.
It can return LZMA_STATUS_* code or LZMA2_PARSE_STATUS_* code:
- LZMA2_PARSE_STATUS_NEW_BLOCK - there is new block, and 1 additional byte (control byte of next block header) was read from input.
- LZMA2_PARSE_STATUS_NEW_CHUNK - there is new chunk, and only lzma2 header of new chunk was read.
CLzma2Dec::unpackSize contains unpack size of that chunk
*/
typedef enum
{
/*
LZMA_STATUS_NOT_SPECIFIED // data error
LZMA_STATUS_FINISHED_WITH_MARK
LZMA_STATUS_NOT_FINISHED //
LZMA_STATUS_NEEDS_MORE_INPUT
LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK // unused
*/
LZMA2_PARSE_STATUS_NEW_BLOCK = LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK + 1,
LZMA2_PARSE_STATUS_NEW_CHUNK
} ELzma2ParseStatus;
ELzma2ParseStatus Lzma2Dec_Parse(CLzma2Dec *p,
SizeT outSize, // output size
const Byte *src, SizeT *srcLen,
int checkFinishBlock // set (checkFinishBlock = 1), if it must read full input data, if decoder.dicPos reaches blockMax position.
);
/*
LZMA2 parser doesn't decode LZMA chunks, so we must read
full input LZMA chunk to decode some part of LZMA chunk.
Lzma2Dec_GetUnpackExtra() returns the value that shows
max possible number of output bytes that can be output by decoder
at current input positon.
*/
#define Lzma2Dec_GetUnpackExtra(p) ((p)->isExtraMode ? (p)->unpackSize : 0);
/* ---------- One Call Interface ---------- */
/*
finishMode:
It has meaning only if the decoding reaches output limit (*destLen).
LZMA_FINISH_ANY - use smallest number of input bytes
LZMA_FINISH_END - read EndOfStream marker after decoding
Returns:
SZ_OK
status:
LZMA_STATUS_FINISHED_WITH_MARK
LZMA_STATUS_NOT_FINISHED
SZ_ERROR_DATA - Data error
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_UNSUPPORTED - Unsupported properties
SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src).
*/
SRes Lzma2Decode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
Byte prop, ELzmaFinishMode finishMode, ELzmaStatus *status, ISzAllocPtr alloc);
EXTERN_C_END
#endif
+1082
View File
File diff suppressed because it is too large Load Diff
+79
View File
@@ -0,0 +1,79 @@
/* Lzma2DecMt.h -- LZMA2 Decoder Multi-thread
2018-02-17 : Igor Pavlov : Public domain */
#ifndef __LZMA2_DEC_MT_H
#define __LZMA2_DEC_MT_H
#include "7zTypes.h"
EXTERN_C_BEGIN
typedef struct
{
size_t inBufSize_ST;
size_t outStep_ST;
#ifndef _7ZIP_ST
unsigned numThreads;
size_t inBufSize_MT;
size_t outBlockMax;
size_t inBlockMax;
#endif
} CLzma2DecMtProps;
/* init to single-thread mode */
void Lzma2DecMtProps_Init(CLzma2DecMtProps *p);
/* ---------- CLzma2DecMtHandle Interface ---------- */
/* Lzma2DecMt_ * functions can return the following exit codes:
SRes:
SZ_OK - OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_PARAM - Incorrect paramater in props
SZ_ERROR_WRITE - ISeqOutStream write callback error
// SZ_ERROR_OUTPUT_EOF - output buffer overflow - version with (Byte *) output
SZ_ERROR_PROGRESS - some break from progress callback
SZ_ERROR_THREAD - error in multithreading functions (only for Mt version)
*/
typedef void * CLzma2DecMtHandle;
CLzma2DecMtHandle Lzma2DecMt_Create(ISzAllocPtr alloc, ISzAllocPtr allocMid);
void Lzma2DecMt_Destroy(CLzma2DecMtHandle p);
SRes Lzma2DecMt_Decode(CLzma2DecMtHandle p,
Byte prop,
const CLzma2DecMtProps *props,
ISeqOutStream *outStream,
const UInt64 *outDataSize, // NULL means undefined
int finishMode, // 0 - partial unpacking is allowed, 1 - if lzma2 stream must be finished
// Byte *outBuf, size_t *outBufSize,
ISeqInStream *inStream,
// const Byte *inData, size_t inDataSize,
// out variables:
UInt64 *inProcessed,
int *isMT, /* out: (*isMT == 0), if single thread decoding was used */
// UInt64 *outProcessed,
ICompressProgress *progress);
/* ---------- Read from CLzma2DecMtHandle Interface ---------- */
SRes Lzma2DecMt_Init(CLzma2DecMtHandle pp,
Byte prop,
const CLzma2DecMtProps *props,
const UInt64 *outDataSize, int finishMode,
ISeqInStream *inStream);
SRes Lzma2DecMt_Read(CLzma2DecMtHandle pp,
Byte *data, size_t *outSize,
UInt64 *inStreamProcessed);
EXTERN_C_END
#endif
+803
View File
@@ -0,0 +1,803 @@
/* Lzma2Enc.c -- LZMA2 Encoder
2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <string.h>
/* #define _7ZIP_ST */
#include "Lzma2Enc.h"
#ifndef _7ZIP_ST
#include "MtCoder.h"
#else
#define MTCODER__THREADS_MAX 1
#endif
#define LZMA2_CONTROL_LZMA (1 << 7)
#define LZMA2_CONTROL_COPY_NO_RESET 2
#define LZMA2_CONTROL_COPY_RESET_DIC 1
#define LZMA2_CONTROL_EOF 0
#define LZMA2_LCLP_MAX 4
#define LZMA2_DIC_SIZE_FROM_PROP(p) (((UInt32)2 | ((p) & 1)) << ((p) / 2 + 11))
#define LZMA2_PACK_SIZE_MAX (1 << 16)
#define LZMA2_COPY_CHUNK_SIZE LZMA2_PACK_SIZE_MAX
#define LZMA2_UNPACK_SIZE_MAX (1 << 21)
#define LZMA2_KEEP_WINDOW_SIZE LZMA2_UNPACK_SIZE_MAX
#define LZMA2_CHUNK_SIZE_COMPRESSED_MAX ((1 << 16) + 16)
#define PRF(x) /* x */
/* ---------- CLimitedSeqInStream ---------- */
typedef struct
{
ISeqInStream vt;
ISeqInStream *realStream;
UInt64 limit;
UInt64 processed;
int finished;
} CLimitedSeqInStream;
static void LimitedSeqInStream_Init(CLimitedSeqInStream *p)
{
p->limit = (UInt64)(Int64)-1;
p->processed = 0;
p->finished = 0;
}
static SRes LimitedSeqInStream_Read(const ISeqInStream *pp, void *data, size_t *size)
{
CLimitedSeqInStream *p = CONTAINER_FROM_VTBL(pp, CLimitedSeqInStream, vt);
size_t size2 = *size;
SRes res = SZ_OK;
if (p->limit != (UInt64)(Int64)-1)
{
UInt64 rem = p->limit - p->processed;
if (size2 > rem)
size2 = (size_t)rem;
}
if (size2 != 0)
{
res = ISeqInStream_Read(p->realStream, data, &size2);
p->finished = (size2 == 0 ? 1 : 0);
p->processed += size2;
}
*size = size2;
return res;
}
/* ---------- CLzma2EncInt ---------- */
typedef struct
{
CLzmaEncHandle enc;
Byte propsAreSet;
Byte propsByte;
Byte needInitState;
Byte needInitProp;
UInt64 srcPos;
} CLzma2EncInt;
static SRes Lzma2EncInt_InitStream(CLzma2EncInt *p, const CLzma2EncProps *props)
{
if (!p->propsAreSet)
{
SizeT propsSize = LZMA_PROPS_SIZE;
Byte propsEncoded[LZMA_PROPS_SIZE];
RINOK(LzmaEnc_SetProps(p->enc, &props->lzmaProps));
RINOK(LzmaEnc_WriteProperties(p->enc, propsEncoded, &propsSize));
p->propsByte = propsEncoded[0];
p->propsAreSet = True;
}
return SZ_OK;
}
static void Lzma2EncInt_InitBlock(CLzma2EncInt *p)
{
p->srcPos = 0;
p->needInitState = True;
p->needInitProp = True;
}
SRes LzmaEnc_PrepareForLzma2(CLzmaEncHandle pp, ISeqInStream *inStream, UInt32 keepWindowSize,
ISzAllocPtr alloc, ISzAllocPtr allocBig);
SRes LzmaEnc_MemPrepare(CLzmaEncHandle pp, const Byte *src, SizeT srcLen,
UInt32 keepWindowSize, ISzAllocPtr alloc, ISzAllocPtr allocBig);
SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32 *unpackSize);
const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle pp);
void LzmaEnc_Finish(CLzmaEncHandle pp);
void LzmaEnc_SaveState(CLzmaEncHandle pp);
void LzmaEnc_RestoreState(CLzmaEncHandle pp);
/*
UInt32 LzmaEnc_GetNumAvailableBytes(CLzmaEncHandle pp);
*/
static SRes Lzma2EncInt_EncodeSubblock(CLzma2EncInt *p, Byte *outBuf,
size_t *packSizeRes, ISeqOutStream *outStream)
{
size_t packSizeLimit = *packSizeRes;
size_t packSize = packSizeLimit;
UInt32 unpackSize = LZMA2_UNPACK_SIZE_MAX;
unsigned lzHeaderSize = 5 + (p->needInitProp ? 1 : 0);
BoolInt useCopyBlock;
SRes res;
*packSizeRes = 0;
if (packSize < lzHeaderSize)
return SZ_ERROR_OUTPUT_EOF;
packSize -= lzHeaderSize;
LzmaEnc_SaveState(p->enc);
res = LzmaEnc_CodeOneMemBlock(p->enc, p->needInitState,
outBuf + lzHeaderSize, &packSize, LZMA2_PACK_SIZE_MAX, &unpackSize);
PRF(printf("\npackSize = %7d unpackSize = %7d ", packSize, unpackSize));
if (unpackSize == 0)
return res;
if (res == SZ_OK)
useCopyBlock = (packSize + 2 >= unpackSize || packSize > (1 << 16));
else
{
if (res != SZ_ERROR_OUTPUT_EOF)
return res;
res = SZ_OK;
useCopyBlock = True;
}
if (useCopyBlock)
{
size_t destPos = 0;
PRF(printf("################# COPY "));
while (unpackSize > 0)
{
UInt32 u = (unpackSize < LZMA2_COPY_CHUNK_SIZE) ? unpackSize : LZMA2_COPY_CHUNK_SIZE;
if (packSizeLimit - destPos < u + 3)
return SZ_ERROR_OUTPUT_EOF;
outBuf[destPos++] = (Byte)(p->srcPos == 0 ? LZMA2_CONTROL_COPY_RESET_DIC : LZMA2_CONTROL_COPY_NO_RESET);
outBuf[destPos++] = (Byte)((u - 1) >> 8);
outBuf[destPos++] = (Byte)(u - 1);
memcpy(outBuf + destPos, LzmaEnc_GetCurBuf(p->enc) - unpackSize, u);
unpackSize -= u;
destPos += u;
p->srcPos += u;
if (outStream)
{
*packSizeRes += destPos;
if (ISeqOutStream_Write(outStream, outBuf, destPos) != destPos)
return SZ_ERROR_WRITE;
destPos = 0;
}
else
*packSizeRes = destPos;
/* needInitState = True; */
}
LzmaEnc_RestoreState(p->enc);
return SZ_OK;
}
{
size_t destPos = 0;
UInt32 u = unpackSize - 1;
UInt32 pm = (UInt32)(packSize - 1);
unsigned mode = (p->srcPos == 0) ? 3 : (p->needInitState ? (p->needInitProp ? 2 : 1) : 0);
PRF(printf(" "));
outBuf[destPos++] = (Byte)(LZMA2_CONTROL_LZMA | (mode << 5) | ((u >> 16) & 0x1F));
outBuf[destPos++] = (Byte)(u >> 8);
outBuf[destPos++] = (Byte)u;
outBuf[destPos++] = (Byte)(pm >> 8);
outBuf[destPos++] = (Byte)pm;
if (p->needInitProp)
outBuf[destPos++] = p->propsByte;
p->needInitProp = False;
p->needInitState = False;
destPos += packSize;
p->srcPos += unpackSize;
if (outStream)
if (ISeqOutStream_Write(outStream, outBuf, destPos) != destPos)
return SZ_ERROR_WRITE;
*packSizeRes = destPos;
return SZ_OK;
}
}
/* ---------- Lzma2 Props ---------- */
void Lzma2EncProps_Init(CLzma2EncProps *p)
{
LzmaEncProps_Init(&p->lzmaProps);
p->blockSize = LZMA2_ENC_PROPS__BLOCK_SIZE__AUTO;
p->numBlockThreads_Reduced = -1;
p->numBlockThreads_Max = -1;
p->numTotalThreads = -1;
}
void Lzma2EncProps_Normalize(CLzma2EncProps *p)
{
UInt64 fileSize;
int t1, t1n, t2, t2r, t3;
{
CLzmaEncProps lzmaProps = p->lzmaProps;
LzmaEncProps_Normalize(&lzmaProps);
t1n = lzmaProps.numThreads;
}
t1 = p->lzmaProps.numThreads;
t2 = p->numBlockThreads_Max;
t3 = p->numTotalThreads;
if (t2 > MTCODER__THREADS_MAX)
t2 = MTCODER__THREADS_MAX;
if (t3 <= 0)
{
if (t2 <= 0)
t2 = 1;
t3 = t1n * t2;
}
else if (t2 <= 0)
{
t2 = t3 / t1n;
if (t2 == 0)
{
t1 = 1;
t2 = t3;
}
if (t2 > MTCODER__THREADS_MAX)
t2 = MTCODER__THREADS_MAX;
}
else if (t1 <= 0)
{
t1 = t3 / t2;
if (t1 == 0)
t1 = 1;
}
else
t3 = t1n * t2;
p->lzmaProps.numThreads = t1;
t2r = t2;
fileSize = p->lzmaProps.reduceSize;
if ( p->blockSize != LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID
&& p->blockSize != LZMA2_ENC_PROPS__BLOCK_SIZE__AUTO
&& (p->blockSize < fileSize || fileSize == (UInt64)(Int64)-1))
p->lzmaProps.reduceSize = p->blockSize;
LzmaEncProps_Normalize(&p->lzmaProps);
p->lzmaProps.reduceSize = fileSize;
t1 = p->lzmaProps.numThreads;
if (p->blockSize == LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID)
{
t2r = t2 = 1;
t3 = t1;
}
else if (p->blockSize == LZMA2_ENC_PROPS__BLOCK_SIZE__AUTO && t2 <= 1)
{
/* if there is no block multi-threading, we use SOLID block */
p->blockSize = LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID;
}
else
{
if (p->blockSize == LZMA2_ENC_PROPS__BLOCK_SIZE__AUTO)
{
const UInt32 kMinSize = (UInt32)1 << 20;
const UInt32 kMaxSize = (UInt32)1 << 28;
const UInt32 dictSize = p->lzmaProps.dictSize;
UInt64 blockSize = (UInt64)dictSize << 2;
if (blockSize < kMinSize) blockSize = kMinSize;
if (blockSize > kMaxSize) blockSize = kMaxSize;
if (blockSize < dictSize) blockSize = dictSize;
blockSize += (kMinSize - 1);
blockSize &= ~(UInt64)(kMinSize - 1);
p->blockSize = blockSize;
}
if (t2 > 1 && fileSize != (UInt64)(Int64)-1)
{
UInt64 numBlocks = fileSize / p->blockSize;
if (numBlocks * p->blockSize != fileSize)
numBlocks++;
if (numBlocks < (unsigned)t2)
{
t2r = (unsigned)numBlocks;
if (t2r == 0)
t2r = 1;
t3 = t1 * t2r;
}
}
}
p->numBlockThreads_Max = t2;
p->numBlockThreads_Reduced = t2r;
p->numTotalThreads = t3;
}
static SRes Progress(ICompressProgress *p, UInt64 inSize, UInt64 outSize)
{
return (p && ICompressProgress_Progress(p, inSize, outSize) != SZ_OK) ? SZ_ERROR_PROGRESS : SZ_OK;
}
/* ---------- Lzma2 ---------- */
typedef struct
{
Byte propEncoded;
CLzma2EncProps props;
UInt64 expectedDataSize;
Byte *tempBufLzma;
ISzAllocPtr alloc;
ISzAllocPtr allocBig;
CLzma2EncInt coders[MTCODER__THREADS_MAX];
#ifndef _7ZIP_ST
ISeqOutStream *outStream;
Byte *outBuf;
size_t outBuf_Rem; /* remainder in outBuf */
size_t outBufSize; /* size of allocated outBufs[i] */
size_t outBufsDataSizes[MTCODER__BLOCKS_MAX];
BoolInt mtCoder_WasConstructed;
CMtCoder mtCoder;
Byte *outBufs[MTCODER__BLOCKS_MAX];
#endif
} CLzma2Enc;
CLzma2EncHandle Lzma2Enc_Create(ISzAllocPtr alloc, ISzAllocPtr allocBig)
{
CLzma2Enc *p = (CLzma2Enc *)ISzAlloc_Alloc(alloc, sizeof(CLzma2Enc));
if (!p)
return NULL;
Lzma2EncProps_Init(&p->props);
Lzma2EncProps_Normalize(&p->props);
p->expectedDataSize = (UInt64)(Int64)-1;
p->tempBufLzma = NULL;
p->alloc = alloc;
p->allocBig = allocBig;
{
unsigned i;
for (i = 0; i < MTCODER__THREADS_MAX; i++)
p->coders[i].enc = NULL;
}
#ifndef _7ZIP_ST
p->mtCoder_WasConstructed = False;
{
unsigned i;
for (i = 0; i < MTCODER__BLOCKS_MAX; i++)
p->outBufs[i] = NULL;
p->outBufSize = 0;
}
#endif
return p;
}
#ifndef _7ZIP_ST
static void Lzma2Enc_FreeOutBufs(CLzma2Enc *p)
{
unsigned i;
for (i = 0; i < MTCODER__BLOCKS_MAX; i++)
if (p->outBufs[i])
{
ISzAlloc_Free(p->alloc, p->outBufs[i]);
p->outBufs[i] = NULL;
}
p->outBufSize = 0;
}
#endif
void Lzma2Enc_Destroy(CLzma2EncHandle pp)
{
CLzma2Enc *p = (CLzma2Enc *)pp;
unsigned i;
for (i = 0; i < MTCODER__THREADS_MAX; i++)
{
CLzma2EncInt *t = &p->coders[i];
if (t->enc)
{
LzmaEnc_Destroy(t->enc, p->alloc, p->allocBig);
t->enc = NULL;
}
}
#ifndef _7ZIP_ST
if (p->mtCoder_WasConstructed)
{
MtCoder_Destruct(&p->mtCoder);
p->mtCoder_WasConstructed = False;
}
Lzma2Enc_FreeOutBufs(p);
#endif
ISzAlloc_Free(p->alloc, p->tempBufLzma);
p->tempBufLzma = NULL;
ISzAlloc_Free(p->alloc, pp);
}
SRes Lzma2Enc_SetProps(CLzma2EncHandle pp, const CLzma2EncProps *props)
{
CLzma2Enc *p = (CLzma2Enc *)pp;
CLzmaEncProps lzmaProps = props->lzmaProps;
LzmaEncProps_Normalize(&lzmaProps);
if (lzmaProps.lc + lzmaProps.lp > LZMA2_LCLP_MAX)
return SZ_ERROR_PARAM;
p->props = *props;
Lzma2EncProps_Normalize(&p->props);
return SZ_OK;
}
void Lzma2Enc_SetDataSize(CLzmaEncHandle pp, UInt64 expectedDataSiize)
{
CLzma2Enc *p = (CLzma2Enc *)pp;
p->expectedDataSize = expectedDataSiize;
}
Byte Lzma2Enc_WriteProperties(CLzma2EncHandle pp)
{
CLzma2Enc *p = (CLzma2Enc *)pp;
unsigned i;
UInt32 dicSize = LzmaEncProps_GetDictSize(&p->props.lzmaProps);
for (i = 0; i < 40; i++)
if (dicSize <= LZMA2_DIC_SIZE_FROM_PROP(i))
break;
return (Byte)i;
}
static SRes Lzma2Enc_EncodeMt1(
CLzma2Enc *me,
CLzma2EncInt *p,
ISeqOutStream *outStream,
Byte *outBuf, size_t *outBufSize,
ISeqInStream *inStream,
const Byte *inData, size_t inDataSize,
int finished,
ICompressProgress *progress)
{
UInt64 unpackTotal = 0;
UInt64 packTotal = 0;
size_t outLim = 0;
CLimitedSeqInStream limitedInStream;
if (outBuf)
{
outLim = *outBufSize;
*outBufSize = 0;
}
if (!p->enc)
{
p->propsAreSet = False;
p->enc = LzmaEnc_Create(me->alloc);
if (!p->enc)
return SZ_ERROR_MEM;
}
limitedInStream.realStream = inStream;
if (inStream)
{
limitedInStream.vt.Read = LimitedSeqInStream_Read;
}
if (!outBuf)
{
// outStream version works only in one thread. So we use CLzma2Enc::tempBufLzma
if (!me->tempBufLzma)
{
me->tempBufLzma = (Byte *)ISzAlloc_Alloc(me->alloc, LZMA2_CHUNK_SIZE_COMPRESSED_MAX);
if (!me->tempBufLzma)
return SZ_ERROR_MEM;
}
}
RINOK(Lzma2EncInt_InitStream(p, &me->props));
for (;;)
{
SRes res = SZ_OK;
size_t inSizeCur = 0;
Lzma2EncInt_InitBlock(p);
LimitedSeqInStream_Init(&limitedInStream);
limitedInStream.limit = me->props.blockSize;
if (inStream)
{
UInt64 expected = (UInt64)(Int64)-1;
// inStream version works only in one thread. So we use CLzma2Enc::expectedDataSize
if (me->expectedDataSize != (UInt64)(Int64)-1
&& me->expectedDataSize >= unpackTotal)
expected = me->expectedDataSize - unpackTotal;
if (me->props.blockSize != LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID
&& expected > me->props.blockSize)
expected = (size_t)me->props.blockSize;
LzmaEnc_SetDataSize(p->enc, expected);
RINOK(LzmaEnc_PrepareForLzma2(p->enc,
&limitedInStream.vt,
LZMA2_KEEP_WINDOW_SIZE,
me->alloc,
me->allocBig));
}
else
{
inSizeCur = inDataSize - (size_t)unpackTotal;
if (me->props.blockSize != LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID
&& inSizeCur > me->props.blockSize)
inSizeCur = (size_t)me->props.blockSize;
// LzmaEnc_SetDataSize(p->enc, inSizeCur);
RINOK(LzmaEnc_MemPrepare(p->enc,
inData + (size_t)unpackTotal, inSizeCur,
LZMA2_KEEP_WINDOW_SIZE,
me->alloc,
me->allocBig));
}
for (;;)
{
size_t packSize = LZMA2_CHUNK_SIZE_COMPRESSED_MAX;
if (outBuf)
packSize = outLim - (size_t)packTotal;
res = Lzma2EncInt_EncodeSubblock(p,
outBuf ? outBuf + (size_t)packTotal : me->tempBufLzma, &packSize,
outBuf ? NULL : outStream);
if (res != SZ_OK)
break;
packTotal += packSize;
if (outBuf)
*outBufSize = (size_t)packTotal;
res = Progress(progress, unpackTotal + p->srcPos, packTotal);
if (res != SZ_OK)
break;
/*
if (LzmaEnc_GetNumAvailableBytes(p->enc) == 0)
break;
*/
if (packSize == 0)
break;
}
LzmaEnc_Finish(p->enc);
unpackTotal += p->srcPos;
RINOK(res);
if (p->srcPos != (inStream ? limitedInStream.processed : inSizeCur))
return SZ_ERROR_FAIL;
if (inStream ? limitedInStream.finished : (unpackTotal == inDataSize))
{
if (finished)
{
if (outBuf)
{
size_t destPos = *outBufSize;
if (destPos >= outLim)
return SZ_ERROR_OUTPUT_EOF;
outBuf[destPos] = 0;
*outBufSize = destPos + 1;
}
else
{
Byte b = 0;
if (ISeqOutStream_Write(outStream, &b, 1) != 1)
return SZ_ERROR_WRITE;
}
}
return SZ_OK;
}
}
}
#ifndef _7ZIP_ST
static SRes Lzma2Enc_MtCallback_Code(void *pp, unsigned coderIndex, unsigned outBufIndex,
const Byte *src, size_t srcSize, int finished)
{
CLzma2Enc *me = (CLzma2Enc *)pp;
size_t destSize = me->outBufSize;
SRes res;
CMtProgressThunk progressThunk;
Byte *dest = me->outBufs[outBufIndex];
me->outBufsDataSizes[outBufIndex] = 0;
if (!dest)
{
dest = (Byte *)ISzAlloc_Alloc(me->alloc, me->outBufSize);
if (!dest)
return SZ_ERROR_MEM;
me->outBufs[outBufIndex] = dest;
}
MtProgressThunk_CreateVTable(&progressThunk);
progressThunk.mtProgress = &me->mtCoder.mtProgress;
progressThunk.inSize = 0;
progressThunk.outSize = 0;
res = Lzma2Enc_EncodeMt1(me,
&me->coders[coderIndex],
NULL, dest, &destSize,
NULL, src, srcSize,
finished,
&progressThunk.vt);
me->outBufsDataSizes[outBufIndex] = destSize;
return res;
}
static SRes Lzma2Enc_MtCallback_Write(void *pp, unsigned outBufIndex)
{
CLzma2Enc *me = (CLzma2Enc *)pp;
size_t size = me->outBufsDataSizes[outBufIndex];
const Byte *data = me->outBufs[outBufIndex];
if (me->outStream)
return ISeqOutStream_Write(me->outStream, data, size) == size ? SZ_OK : SZ_ERROR_WRITE;
if (size > me->outBuf_Rem)
return SZ_ERROR_OUTPUT_EOF;
memcpy(me->outBuf, data, size);
me->outBuf_Rem -= size;
me->outBuf += size;
return SZ_OK;
}
#endif
SRes Lzma2Enc_Encode2(CLzma2EncHandle pp,
ISeqOutStream *outStream,
Byte *outBuf, size_t *outBufSize,
ISeqInStream *inStream,
const Byte *inData, size_t inDataSize,
ICompressProgress *progress)
{
CLzma2Enc *p = (CLzma2Enc *)pp;
if (inStream && inData)
return SZ_ERROR_PARAM;
if (outStream && outBuf)
return SZ_ERROR_PARAM;
{
unsigned i;
for (i = 0; i < MTCODER__THREADS_MAX; i++)
p->coders[i].propsAreSet = False;
}
#ifndef _7ZIP_ST
if (p->props.numBlockThreads_Reduced > 1)
{
IMtCoderCallback2 vt;
if (!p->mtCoder_WasConstructed)
{
p->mtCoder_WasConstructed = True;
MtCoder_Construct(&p->mtCoder);
}
vt.Code = Lzma2Enc_MtCallback_Code;
vt.Write = Lzma2Enc_MtCallback_Write;
p->outStream = outStream;
p->outBuf = NULL;
p->outBuf_Rem = 0;
if (!outStream)
{
p->outBuf = outBuf;
p->outBuf_Rem = *outBufSize;
*outBufSize = 0;
}
p->mtCoder.allocBig = p->allocBig;
p->mtCoder.progress = progress;
p->mtCoder.inStream = inStream;
p->mtCoder.inData = inData;
p->mtCoder.inDataSize = inDataSize;
p->mtCoder.mtCallback = &vt;
p->mtCoder.mtCallbackObject = p;
p->mtCoder.blockSize = (size_t)p->props.blockSize;
if (p->mtCoder.blockSize != p->props.blockSize)
return SZ_ERROR_PARAM; /* SZ_ERROR_MEM */
{
size_t destBlockSize = p->mtCoder.blockSize + (p->mtCoder.blockSize >> 10) + 16;
if (destBlockSize < p->mtCoder.blockSize)
return SZ_ERROR_PARAM;
if (p->outBufSize != destBlockSize)
Lzma2Enc_FreeOutBufs(p);
p->outBufSize = destBlockSize;
}
p->mtCoder.numThreadsMax = p->props.numBlockThreads_Max;
p->mtCoder.expectedDataSize = p->expectedDataSize;
{
SRes res = MtCoder_Code(&p->mtCoder);
if (!outStream)
*outBufSize = p->outBuf - outBuf;
return res;
}
}
#endif
return Lzma2Enc_EncodeMt1(p,
&p->coders[0],
outStream, outBuf, outBufSize,
inStream, inData, inDataSize,
True, /* finished */
progress);
}
+55
View File
@@ -0,0 +1,55 @@
/* Lzma2Enc.h -- LZMA2 Encoder
2017-07-27 : Igor Pavlov : Public domain */
#ifndef __LZMA2_ENC_H
#define __LZMA2_ENC_H
#include "LzmaEnc.h"
EXTERN_C_BEGIN
#define LZMA2_ENC_PROPS__BLOCK_SIZE__AUTO 0
#define LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID ((UInt64)(Int64)-1)
typedef struct
{
CLzmaEncProps lzmaProps;
UInt64 blockSize;
int numBlockThreads_Reduced;
int numBlockThreads_Max;
int numTotalThreads;
} CLzma2EncProps;
void Lzma2EncProps_Init(CLzma2EncProps *p);
void Lzma2EncProps_Normalize(CLzma2EncProps *p);
/* ---------- CLzmaEnc2Handle Interface ---------- */
/* Lzma2Enc_* functions can return the following exit codes:
SRes:
SZ_OK - OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_PARAM - Incorrect paramater in props
SZ_ERROR_WRITE - ISeqOutStream write callback error
SZ_ERROR_OUTPUT_EOF - output buffer overflow - version with (Byte *) output
SZ_ERROR_PROGRESS - some break from progress callback
SZ_ERROR_THREAD - error in multithreading functions (only for Mt version)
*/
typedef void * CLzma2EncHandle;
CLzma2EncHandle Lzma2Enc_Create(ISzAllocPtr alloc, ISzAllocPtr allocBig);
void Lzma2Enc_Destroy(CLzma2EncHandle p);
SRes Lzma2Enc_SetProps(CLzma2EncHandle p, const CLzma2EncProps *props);
void Lzma2Enc_SetDataSize(CLzma2EncHandle p, UInt64 expectedDataSiize);
Byte Lzma2Enc_WriteProperties(CLzma2EncHandle p);
SRes Lzma2Enc_Encode2(CLzma2EncHandle p,
ISeqOutStream *outStream,
Byte *outBuf, size_t *outBufSize,
ISeqInStream *inStream,
const Byte *inData, size_t inDataSize,
ICompressProgress *progress);
EXTERN_C_END
#endif
+111
View File
@@ -0,0 +1,111 @@
/* Lzma86.h -- LZMA + x86 (BCJ) Filter
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __LZMA86_H
#define __LZMA86_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define LZMA86_SIZE_OFFSET (1 + 5)
#define LZMA86_HEADER_SIZE (LZMA86_SIZE_OFFSET + 8)
/*
It's an example for LZMA + x86 Filter use.
You can use .lzma86 extension, if you write that stream to file.
.lzma86 header adds one additional byte to standard .lzma header.
.lzma86 header (14 bytes):
Offset Size Description
0 1 = 0 - no filter, pure LZMA
= 1 - x86 filter + LZMA
1 1 lc, lp and pb in encoded form
2 4 dictSize (little endian)
6 8 uncompressed size (little endian)
Lzma86_Encode
-------------
level - compression level: 0 <= level <= 9, the default value for "level" is 5.
dictSize - The dictionary size in bytes. The maximum value is
128 MB = (1 << 27) bytes for 32-bit version
1 GB = (1 << 30) bytes for 64-bit version
The default value is 16 MB = (1 << 24) bytes, for level = 5.
It's recommended to use the dictionary that is larger than 4 KB and
that can be calculated as (1 << N) or (3 << N) sizes.
For better compression ratio dictSize must be >= inSize.
filterMode:
SZ_FILTER_NO - no Filter
SZ_FILTER_YES - x86 Filter
SZ_FILTER_AUTO - it tries both alternatives to select best.
Encoder will use 2 or 3 passes:
2 passes when FILTER_NO provides better compression.
3 passes when FILTER_YES provides better compression.
Lzma86Encode allocates Data with MyAlloc functions.
RAM Requirements for compressing:
RamSize = dictionarySize * 11.5 + 6MB + FilterBlockSize
filterMode FilterBlockSize
SZ_FILTER_NO 0
SZ_FILTER_YES inSize
SZ_FILTER_AUTO inSize
Return code:
SZ_OK - OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_PARAM - Incorrect paramater
SZ_ERROR_OUTPUT_EOF - output buffer overflow
SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version)
*/
enum ESzFilterMode
{
SZ_FILTER_NO,
SZ_FILTER_YES,
SZ_FILTER_AUTO
};
SRes Lzma86_Encode(Byte *dest, size_t *destLen, const Byte *src, size_t srcLen,
int level, UInt32 dictSize, int filterMode);
/*
Lzma86_GetUnpackSize:
In:
src - input data
srcLen - input data size
Out:
unpackSize - size of uncompressed stream
Return code:
SZ_OK - OK
SZ_ERROR_INPUT_EOF - Error in headers
*/
SRes Lzma86_GetUnpackSize(const Byte *src, SizeT srcLen, UInt64 *unpackSize);
/*
Lzma86_Decode:
In:
dest - output data
destLen - output data size
src - input data
srcLen - input data size
Out:
destLen - processed output size
srcLen - processed input size
Return code:
SZ_OK - OK
SZ_ERROR_DATA - Data error
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_UNSUPPORTED - unsupported file
SZ_ERROR_INPUT_EOF - it needs more bytes in input buffer
*/
SRes Lzma86_Decode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen);
EXTERN_C_END
#endif
+54
View File
@@ -0,0 +1,54 @@
/* Lzma86Dec.c -- LZMA + x86 (BCJ) Filter Decoder
2016-05-16 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "Lzma86.h"
#include "Alloc.h"
#include "Bra.h"
#include "LzmaDec.h"
SRes Lzma86_GetUnpackSize(const Byte *src, SizeT srcLen, UInt64 *unpackSize)
{
unsigned i;
if (srcLen < LZMA86_HEADER_SIZE)
return SZ_ERROR_INPUT_EOF;
*unpackSize = 0;
for (i = 0; i < sizeof(UInt64); i++)
*unpackSize += ((UInt64)src[LZMA86_SIZE_OFFSET + i]) << (8 * i);
return SZ_OK;
}
SRes Lzma86_Decode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen)
{
SRes res;
int useFilter;
SizeT inSizePure;
ELzmaStatus status;
if (*srcLen < LZMA86_HEADER_SIZE)
return SZ_ERROR_INPUT_EOF;
useFilter = src[0];
if (useFilter > 1)
{
*destLen = 0;
return SZ_ERROR_UNSUPPORTED;
}
inSizePure = *srcLen - LZMA86_HEADER_SIZE;
res = LzmaDecode(dest, destLen, src + LZMA86_HEADER_SIZE, &inSizePure,
src + 1, LZMA_PROPS_SIZE, LZMA_FINISH_ANY, &status, &g_Alloc);
*srcLen = inSizePure + LZMA86_HEADER_SIZE;
if (res != SZ_OK)
return res;
if (useFilter == 1)
{
UInt32 x86State;
x86_Convert_Init(x86State);
x86_Convert(dest, *destLen, 0, &x86State, 0);
}
return SZ_OK;
}
+106
View File
@@ -0,0 +1,106 @@
/* Lzma86Enc.c -- LZMA + x86 (BCJ) Filter Encoder
2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <string.h>
#include "Lzma86.h"
#include "Alloc.h"
#include "Bra.h"
#include "LzmaEnc.h"
#define SZE_OUT_OVERFLOW SZE_DATA_ERROR
int Lzma86_Encode(Byte *dest, size_t *destLen, const Byte *src, size_t srcLen,
int level, UInt32 dictSize, int filterMode)
{
size_t outSize2 = *destLen;
Byte *filteredStream;
BoolInt useFilter;
int mainResult = SZ_ERROR_OUTPUT_EOF;
CLzmaEncProps props;
LzmaEncProps_Init(&props);
props.level = level;
props.dictSize = dictSize;
*destLen = 0;
if (outSize2 < LZMA86_HEADER_SIZE)
return SZ_ERROR_OUTPUT_EOF;
{
int i;
UInt64 t = srcLen;
for (i = 0; i < 8; i++, t >>= 8)
dest[LZMA86_SIZE_OFFSET + i] = (Byte)t;
}
filteredStream = 0;
useFilter = (filterMode != SZ_FILTER_NO);
if (useFilter)
{
if (srcLen != 0)
{
filteredStream = (Byte *)MyAlloc(srcLen);
if (filteredStream == 0)
return SZ_ERROR_MEM;
memcpy(filteredStream, src, srcLen);
}
{
UInt32 x86State;
x86_Convert_Init(x86State);
x86_Convert(filteredStream, srcLen, 0, &x86State, 1);
}
}
{
size_t minSize = 0;
BoolInt bestIsFiltered = False;
/* passes for SZ_FILTER_AUTO:
0 - BCJ + LZMA
1 - LZMA
2 - BCJ + LZMA agaian, if pass 0 (BCJ + LZMA) is better.
*/
int numPasses = (filterMode == SZ_FILTER_AUTO) ? 3 : 1;
int i;
for (i = 0; i < numPasses; i++)
{
size_t outSizeProcessed = outSize2 - LZMA86_HEADER_SIZE;
size_t outPropsSize = 5;
SRes curRes;
BoolInt curModeIsFiltered = (numPasses > 1 && i == numPasses - 1);
if (curModeIsFiltered && !bestIsFiltered)
break;
if (useFilter && i == 0)
curModeIsFiltered = True;
curRes = LzmaEncode(dest + LZMA86_HEADER_SIZE, &outSizeProcessed,
curModeIsFiltered ? filteredStream : src, srcLen,
&props, dest + 1, &outPropsSize, 0,
NULL, &g_Alloc, &g_Alloc);
if (curRes != SZ_ERROR_OUTPUT_EOF)
{
if (curRes != SZ_OK)
{
mainResult = curRes;
break;
}
if (outSizeProcessed <= minSize || mainResult != SZ_OK)
{
minSize = outSizeProcessed;
bestIsFiltered = curModeIsFiltered;
mainResult = SZ_OK;
}
}
}
dest[0] = (Byte)(bestIsFiltered ? 1 : 0);
*destLen = LZMA86_HEADER_SIZE + minSize;
}
if (useFilter)
MyFree(filteredStream);
return mainResult;
}
+1185
View File
File diff suppressed because it is too large Load Diff
+234
View File
@@ -0,0 +1,234 @@
/* LzmaDec.h -- LZMA Decoder
2018-04-21 : Igor Pavlov : Public domain */
#ifndef __LZMA_DEC_H
#define __LZMA_DEC_H
#include "7zTypes.h"
EXTERN_C_BEGIN
/* #define _LZMA_PROB32 */
/* _LZMA_PROB32 can increase the speed on some CPUs,
but memory usage for CLzmaDec::probs will be doubled in that case */
typedef
#ifdef _LZMA_PROB32
UInt32
#else
UInt16
#endif
CLzmaProb;
/* ---------- LZMA Properties ---------- */
#define LZMA_PROPS_SIZE 5
typedef struct _CLzmaProps
{
Byte lc;
Byte lp;
Byte pb;
Byte _pad_;
UInt32 dicSize;
} CLzmaProps;
/* LzmaProps_Decode - decodes properties
Returns:
SZ_OK
SZ_ERROR_UNSUPPORTED - Unsupported properties
*/
SRes LzmaProps_Decode(CLzmaProps *p, const Byte *data, unsigned size);
/* ---------- LZMA Decoder state ---------- */
/* LZMA_REQUIRED_INPUT_MAX = number of required input bytes for worst case.
Num bits = log2((2^11 / 31) ^ 22) + 26 < 134 + 26 = 160; */
#define LZMA_REQUIRED_INPUT_MAX 20
typedef struct
{
/* Don't change this structure. ASM code can use it. */
CLzmaProps prop;
CLzmaProb *probs;
CLzmaProb *probs_1664;
Byte *dic;
SizeT dicBufSize;
SizeT dicPos;
const Byte *buf;
UInt32 range;
UInt32 code;
UInt32 processedPos;
UInt32 checkDicSize;
UInt32 reps[4];
UInt32 state;
UInt32 remainLen;
UInt32 numProbs;
unsigned tempBufSize;
Byte tempBuf[LZMA_REQUIRED_INPUT_MAX];
} CLzmaDec;
#define LzmaDec_Construct(p) { (p)->dic = NULL; (p)->probs = NULL; }
void LzmaDec_Init(CLzmaDec *p);
/* There are two types of LZMA streams:
- Stream with end mark. That end mark adds about 6 bytes to compressed size.
- Stream without end mark. You must know exact uncompressed size to decompress such stream. */
typedef enum
{
LZMA_FINISH_ANY, /* finish at any point */
LZMA_FINISH_END /* block must be finished at the end */
} ELzmaFinishMode;
/* ELzmaFinishMode has meaning only if the decoding reaches output limit !!!
You must use LZMA_FINISH_END, when you know that current output buffer
covers last bytes of block. In other cases you must use LZMA_FINISH_ANY.
If LZMA decoder sees end marker before reaching output limit, it returns SZ_OK,
and output value of destLen will be less than output buffer size limit.
You can check status result also.
You can use multiple checks to test data integrity after full decompression:
1) Check Result and "status" variable.
2) Check that output(destLen) = uncompressedSize, if you know real uncompressedSize.
3) Check that output(srcLen) = compressedSize, if you know real compressedSize.
You must use correct finish mode in that case. */
typedef enum
{
LZMA_STATUS_NOT_SPECIFIED, /* use main error code instead */
LZMA_STATUS_FINISHED_WITH_MARK, /* stream was finished with end mark. */
LZMA_STATUS_NOT_FINISHED, /* stream was not finished */
LZMA_STATUS_NEEDS_MORE_INPUT, /* you must provide more input bytes */
LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK /* there is probability that stream was finished without end mark */
} ELzmaStatus;
/* ELzmaStatus is used only as output value for function call */
/* ---------- Interfaces ---------- */
/* There are 3 levels of interfaces:
1) Dictionary Interface
2) Buffer Interface
3) One Call Interface
You can select any of these interfaces, but don't mix functions from different
groups for same object. */
/* There are two variants to allocate state for Dictionary Interface:
1) LzmaDec_Allocate / LzmaDec_Free
2) LzmaDec_AllocateProbs / LzmaDec_FreeProbs
You can use variant 2, if you set dictionary buffer manually.
For Buffer Interface you must always use variant 1.
LzmaDec_Allocate* can return:
SZ_OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_UNSUPPORTED - Unsupported properties
*/
SRes LzmaDec_AllocateProbs(CLzmaDec *p, const Byte *props, unsigned propsSize, ISzAllocPtr alloc);
void LzmaDec_FreeProbs(CLzmaDec *p, ISzAllocPtr alloc);
SRes LzmaDec_Allocate(CLzmaDec *p, const Byte *props, unsigned propsSize, ISzAllocPtr alloc);
void LzmaDec_Free(CLzmaDec *p, ISzAllocPtr alloc);
/* ---------- Dictionary Interface ---------- */
/* You can use it, if you want to eliminate the overhead for data copying from
dictionary to some other external buffer.
You must work with CLzmaDec variables directly in this interface.
STEPS:
LzmaDec_Construct()
LzmaDec_Allocate()
for (each new stream)
{
LzmaDec_Init()
while (it needs more decompression)
{
LzmaDec_DecodeToDic()
use data from CLzmaDec::dic and update CLzmaDec::dicPos
}
}
LzmaDec_Free()
*/
/* LzmaDec_DecodeToDic
The decoding to internal dictionary buffer (CLzmaDec::dic).
You must manually update CLzmaDec::dicPos, if it reaches CLzmaDec::dicBufSize !!!
finishMode:
It has meaning only if the decoding reaches output limit (dicLimit).
LZMA_FINISH_ANY - Decode just dicLimit bytes.
LZMA_FINISH_END - Stream must be finished after dicLimit.
Returns:
SZ_OK
status:
LZMA_STATUS_FINISHED_WITH_MARK
LZMA_STATUS_NOT_FINISHED
LZMA_STATUS_NEEDS_MORE_INPUT
LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK
SZ_ERROR_DATA - Data error
*/
SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit,
const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);
/* ---------- Buffer Interface ---------- */
/* It's zlib-like interface.
See LzmaDec_DecodeToDic description for information about STEPS and return results,
but you must use LzmaDec_DecodeToBuf instead of LzmaDec_DecodeToDic and you don't need
to work with CLzmaDec variables manually.
finishMode:
It has meaning only if the decoding reaches output limit (*destLen).
LZMA_FINISH_ANY - Decode just destLen bytes.
LZMA_FINISH_END - Stream must be finished after (*destLen).
*/
SRes LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen,
const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);
/* ---------- One Call Interface ---------- */
/* LzmaDecode
finishMode:
It has meaning only if the decoding reaches output limit (*destLen).
LZMA_FINISH_ANY - Decode just destLen bytes.
LZMA_FINISH_END - Stream must be finished after (*destLen).
Returns:
SZ_OK
status:
LZMA_STATUS_FINISHED_WITH_MARK
LZMA_STATUS_NOT_FINISHED
LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK
SZ_ERROR_DATA - Data error
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_UNSUPPORTED - Unsupported properties
SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src).
*/
SRes LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode,
ELzmaStatus *status, ISzAllocPtr alloc);
EXTERN_C_END
#endif
+2976
View File
File diff suppressed because it is too large Load Diff
+76
View File
@@ -0,0 +1,76 @@
/* LzmaEnc.h -- LZMA Encoder
2017-07-27 : Igor Pavlov : Public domain */
#ifndef __LZMA_ENC_H
#define __LZMA_ENC_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define LZMA_PROPS_SIZE 5
typedef struct _CLzmaEncProps
{
int level; /* 0 <= level <= 9 */
UInt32 dictSize; /* (1 << 12) <= dictSize <= (1 << 27) for 32-bit version
(1 << 12) <= dictSize <= (3 << 29) for 64-bit version
default = (1 << 24) */
int lc; /* 0 <= lc <= 8, default = 3 */
int lp; /* 0 <= lp <= 4, default = 0 */
int pb; /* 0 <= pb <= 4, default = 2 */
int algo; /* 0 - fast, 1 - normal, default = 1 */
int fb; /* 5 <= fb <= 273, default = 32 */
int btMode; /* 0 - hashChain Mode, 1 - binTree mode - normal, default = 1 */
int numHashBytes; /* 2, 3 or 4, default = 4 */
UInt32 mc; /* 1 <= mc <= (1 << 30), default = 32 */
unsigned writeEndMark; /* 0 - do not write EOPM, 1 - write EOPM, default = 0 */
int numThreads; /* 1 or 2, default = 2 */
UInt64 reduceSize; /* estimated size of data that will be compressed. default = (UInt64)(Int64)-1.
Encoder uses this value to reduce dictionary size */
} CLzmaEncProps;
void LzmaEncProps_Init(CLzmaEncProps *p);
void LzmaEncProps_Normalize(CLzmaEncProps *p);
UInt32 LzmaEncProps_GetDictSize(const CLzmaEncProps *props2);
/* ---------- CLzmaEncHandle Interface ---------- */
/* LzmaEnc* functions can return the following exit codes:
SRes:
SZ_OK - OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_PARAM - Incorrect paramater in props
SZ_ERROR_WRITE - ISeqOutStream write callback error
SZ_ERROR_OUTPUT_EOF - output buffer overflow - version with (Byte *) output
SZ_ERROR_PROGRESS - some break from progress callback
SZ_ERROR_THREAD - error in multithreading functions (only for Mt version)
*/
typedef void * CLzmaEncHandle;
CLzmaEncHandle LzmaEnc_Create(ISzAllocPtr alloc);
void LzmaEnc_Destroy(CLzmaEncHandle p, ISzAllocPtr alloc, ISzAllocPtr allocBig);
SRes LzmaEnc_SetProps(CLzmaEncHandle p, const CLzmaEncProps *props);
void LzmaEnc_SetDataSize(CLzmaEncHandle p, UInt64 expectedDataSiize);
SRes LzmaEnc_WriteProperties(CLzmaEncHandle p, Byte *properties, SizeT *size);
unsigned LzmaEnc_IsWriteEndMark(CLzmaEncHandle p);
SRes LzmaEnc_Encode(CLzmaEncHandle p, ISeqOutStream *outStream, ISeqInStream *inStream,
ICompressProgress *progress, ISzAllocPtr alloc, ISzAllocPtr allocBig);
SRes LzmaEnc_MemEncode(CLzmaEncHandle p, Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
int writeEndMark, ICompressProgress *progress, ISzAllocPtr alloc, ISzAllocPtr allocBig);
/* ---------- One Call Interface ---------- */
SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeEndMark,
ICompressProgress *progress, ISzAllocPtr alloc, ISzAllocPtr allocBig);
EXTERN_C_END
#endif
+40
View File
@@ -0,0 +1,40 @@
/* LzmaLib.c -- LZMA library wrapper
2015-06-13 : Igor Pavlov : Public domain */
#include "Alloc.h"
#include "LzmaDec.h"
#include "LzmaEnc.h"
#include "LzmaLib.h"
MY_STDAPI LzmaCompress(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t srcLen,
unsigned char *outProps, size_t *outPropsSize,
int level, /* 0 <= level <= 9, default = 5 */
unsigned dictSize, /* use (1 << N) or (3 << N). 4 KB < dictSize <= 128 MB */
int lc, /* 0 <= lc <= 8, default = 3 */
int lp, /* 0 <= lp <= 4, default = 0 */
int pb, /* 0 <= pb <= 4, default = 2 */
int fb, /* 5 <= fb <= 273, default = 32 */
int numThreads /* 1 or 2, default = 2 */
)
{
CLzmaEncProps props;
LzmaEncProps_Init(&props);
props.level = level;
props.dictSize = dictSize;
props.lc = lc;
props.lp = lp;
props.pb = pb;
props.fb = fb;
props.numThreads = numThreads;
return LzmaEncode(dest, destLen, src, srcLen, &props, outProps, outPropsSize, 0,
NULL, &g_Alloc, &g_Alloc);
}
MY_STDAPI LzmaUncompress(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t *srcLen,
const unsigned char *props, size_t propsSize)
{
ELzmaStatus status;
return LzmaDecode(dest, destLen, src, srcLen, props, (unsigned)propsSize, LZMA_FINISH_ANY, &status, &g_Alloc);
}
+131
View File
@@ -0,0 +1,131 @@
/* LzmaLib.h -- LZMA library interface
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __LZMA_LIB_H
#define __LZMA_LIB_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define MY_STDAPI int MY_STD_CALL
#define LZMA_PROPS_SIZE 5
/*
RAM requirements for LZMA:
for compression: (dictSize * 11.5 + 6 MB) + state_size
for decompression: dictSize + state_size
state_size = (4 + (1.5 << (lc + lp))) KB
by default (lc=3, lp=0), state_size = 16 KB.
LZMA properties (5 bytes) format
Offset Size Description
0 1 lc, lp and pb in encoded form.
1 4 dictSize (little endian).
*/
/*
LzmaCompress
------------
outPropsSize -
In: the pointer to the size of outProps buffer; *outPropsSize = LZMA_PROPS_SIZE = 5.
Out: the pointer to the size of written properties in outProps buffer; *outPropsSize = LZMA_PROPS_SIZE = 5.
LZMA Encoder will use defult values for any parameter, if it is
-1 for any from: level, loc, lp, pb, fb, numThreads
0 for dictSize
level - compression level: 0 <= level <= 9;
level dictSize algo fb
0: 16 KB 0 32
1: 64 KB 0 32
2: 256 KB 0 32
3: 1 MB 0 32
4: 4 MB 0 32
5: 16 MB 1 32
6: 32 MB 1 32
7+: 64 MB 1 64
The default value for "level" is 5.
algo = 0 means fast method
algo = 1 means normal method
dictSize - The dictionary size in bytes. The maximum value is
128 MB = (1 << 27) bytes for 32-bit version
1 GB = (1 << 30) bytes for 64-bit version
The default value is 16 MB = (1 << 24) bytes.
It's recommended to use the dictionary that is larger than 4 KB and
that can be calculated as (1 << N) or (3 << N) sizes.
lc - The number of literal context bits (high bits of previous literal).
It can be in the range from 0 to 8. The default value is 3.
Sometimes lc=4 gives the gain for big files.
lp - The number of literal pos bits (low bits of current position for literals).
It can be in the range from 0 to 4. The default value is 0.
The lp switch is intended for periodical data when the period is equal to 2^lp.
For example, for 32-bit (4 bytes) periodical data you can use lp=2. Often it's
better to set lc=0, if you change lp switch.
pb - The number of pos bits (low bits of current position).
It can be in the range from 0 to 4. The default value is 2.
The pb switch is intended for periodical data when the period is equal 2^pb.
fb - Word size (the number of fast bytes).
It can be in the range from 5 to 273. The default value is 32.
Usually, a big number gives a little bit better compression ratio and
slower compression process.
numThreads - The number of thereads. 1 or 2. The default value is 2.
Fast mode (algo = 0) can use only 1 thread.
Out:
destLen - processed output size
Returns:
SZ_OK - OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_PARAM - Incorrect paramater
SZ_ERROR_OUTPUT_EOF - output buffer overflow
SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version)
*/
MY_STDAPI LzmaCompress(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t srcLen,
unsigned char *outProps, size_t *outPropsSize, /* *outPropsSize must be = 5 */
int level, /* 0 <= level <= 9, default = 5 */
unsigned dictSize, /* default = (1 << 24) */
int lc, /* 0 <= lc <= 8, default = 3 */
int lp, /* 0 <= lp <= 4, default = 0 */
int pb, /* 0 <= pb <= 4, default = 2 */
int fb, /* 5 <= fb <= 273, default = 32 */
int numThreads /* 1 or 2, default = 2 */
);
/*
LzmaUncompress
--------------
In:
dest - output data
destLen - output data size
src - input data
srcLen - input data size
Out:
destLen - processed output size
srcLen - processed input size
Returns:
SZ_OK - OK
SZ_ERROR_DATA - Data error
SZ_ERROR_MEM - Memory allocation arror
SZ_ERROR_UNSUPPORTED - Unsupported properties
SZ_ERROR_INPUT_EOF - it needs more bytes in input buffer (src)
*/
MY_STDAPI LzmaUncompress(unsigned char *dest, size_t *destLen, const unsigned char *src, SizeT *srcLen,
const unsigned char *props, size_t propsSize);
EXTERN_C_END
#endif
+601
View File
@@ -0,0 +1,601 @@
/* MtCoder.c -- Multi-thread Coder
2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "MtCoder.h"
#ifndef _7ZIP_ST
SRes MtProgressThunk_Progress(const ICompressProgress *pp, UInt64 inSize, UInt64 outSize)
{
CMtProgressThunk *thunk = CONTAINER_FROM_VTBL(pp, CMtProgressThunk, vt);
UInt64 inSize2 = 0;
UInt64 outSize2 = 0;
if (inSize != (UInt64)(Int64)-1)
{
inSize2 = inSize - thunk->inSize;
thunk->inSize = inSize;
}
if (outSize != (UInt64)(Int64)-1)
{
outSize2 = outSize - thunk->outSize;
thunk->outSize = outSize;
}
return MtProgress_ProgressAdd(thunk->mtProgress, inSize2, outSize2);
}
void MtProgressThunk_CreateVTable(CMtProgressThunk *p)
{
p->vt.Progress = MtProgressThunk_Progress;
}
#define RINOK_THREAD(x) { if ((x) != 0) return SZ_ERROR_THREAD; }
static WRes ArEvent_OptCreate_And_Reset(CEvent *p)
{
if (Event_IsCreated(p))
return Event_Reset(p);
return AutoResetEvent_CreateNotSignaled(p);
}
static THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE ThreadFunc(void *pp);
static SRes MtCoderThread_CreateAndStart(CMtCoderThread *t)
{
WRes wres = ArEvent_OptCreate_And_Reset(&t->startEvent);
if (wres == 0)
{
t->stop = False;
if (!Thread_WasCreated(&t->thread))
wres = Thread_Create(&t->thread, ThreadFunc, t);
if (wres == 0)
wres = Event_Set(&t->startEvent);
}
if (wres == 0)
return SZ_OK;
return MY_SRes_HRESULT_FROM_WRes(wres);
}
static void MtCoderThread_Destruct(CMtCoderThread *t)
{
if (Thread_WasCreated(&t->thread))
{
t->stop = 1;
Event_Set(&t->startEvent);
Thread_Wait(&t->thread);
Thread_Close(&t->thread);
}
Event_Close(&t->startEvent);
if (t->inBuf)
{
ISzAlloc_Free(t->mtCoder->allocBig, t->inBuf);
t->inBuf = NULL;
}
}
static SRes FullRead(ISeqInStream *stream, Byte *data, size_t *processedSize)
{
size_t size = *processedSize;
*processedSize = 0;
while (size != 0)
{
size_t cur = size;
SRes res = ISeqInStream_Read(stream, data, &cur);
*processedSize += cur;
data += cur;
size -= cur;
RINOK(res);
if (cur == 0)
return SZ_OK;
}
return SZ_OK;
}
/*
ThreadFunc2() returns:
SZ_OK - in all normal cases (even for stream error or memory allocation error)
SZ_ERROR_THREAD - in case of failure in system synch function
*/
static SRes ThreadFunc2(CMtCoderThread *t)
{
CMtCoder *mtc = t->mtCoder;
for (;;)
{
unsigned bi;
SRes res;
SRes res2;
BoolInt finished;
unsigned bufIndex;
size_t size;
const Byte *inData;
UInt64 readProcessed = 0;
RINOK_THREAD(Event_Wait(&mtc->readEvent))
/* after Event_Wait(&mtc->readEvent) we must call Event_Set(&mtc->readEvent) in any case to unlock another threads */
if (mtc->stopReading)
{
return Event_Set(&mtc->readEvent) == 0 ? SZ_OK : SZ_ERROR_THREAD;
}
res = MtProgress_GetError(&mtc->mtProgress);
size = 0;
inData = NULL;
finished = True;
if (res == SZ_OK)
{
size = mtc->blockSize;
if (mtc->inStream)
{
if (!t->inBuf)
{
t->inBuf = (Byte *)ISzAlloc_Alloc(mtc->allocBig, mtc->blockSize);
if (!t->inBuf)
res = SZ_ERROR_MEM;
}
if (res == SZ_OK)
{
res = FullRead(mtc->inStream, t->inBuf, &size);
readProcessed = mtc->readProcessed + size;
mtc->readProcessed = readProcessed;
}
if (res != SZ_OK)
{
mtc->readRes = res;
/* after reading error - we can stop encoding of previous blocks */
MtProgress_SetError(&mtc->mtProgress, res);
}
else
finished = (size != mtc->blockSize);
}
else
{
size_t rem;
readProcessed = mtc->readProcessed;
rem = mtc->inDataSize - (size_t)readProcessed;
if (size > rem)
size = rem;
inData = mtc->inData + (size_t)readProcessed;
readProcessed += size;
mtc->readProcessed = readProcessed;
finished = (mtc->inDataSize == (size_t)readProcessed);
}
}
/* we must get some block from blocksSemaphore before Event_Set(&mtc->readEvent) */
res2 = SZ_OK;
if (Semaphore_Wait(&mtc->blocksSemaphore) != 0)
{
res2 = SZ_ERROR_THREAD;
if (res == SZ_OK)
{
res = res2;
// MtProgress_SetError(&mtc->mtProgress, res);
}
}
bi = mtc->blockIndex;
if (++mtc->blockIndex >= mtc->numBlocksMax)
mtc->blockIndex = 0;
bufIndex = (unsigned)(int)-1;
if (res == SZ_OK)
res = MtProgress_GetError(&mtc->mtProgress);
if (res != SZ_OK)
finished = True;
if (!finished)
{
if (mtc->numStartedThreads < mtc->numStartedThreadsLimit
&& mtc->expectedDataSize != readProcessed)
{
res = MtCoderThread_CreateAndStart(&mtc->threads[mtc->numStartedThreads]);
if (res == SZ_OK)
mtc->numStartedThreads++;
else
{
MtProgress_SetError(&mtc->mtProgress, res);
finished = True;
}
}
}
if (finished)
mtc->stopReading = True;
RINOK_THREAD(Event_Set(&mtc->readEvent))
if (res2 != SZ_OK)
return res2;
if (res == SZ_OK)
{
CriticalSection_Enter(&mtc->cs);
bufIndex = mtc->freeBlockHead;
mtc->freeBlockHead = mtc->freeBlockList[bufIndex];
CriticalSection_Leave(&mtc->cs);
res = mtc->mtCallback->Code(mtc->mtCallbackObject, t->index, bufIndex,
mtc->inStream ? t->inBuf : inData, size, finished);
// MtProgress_Reinit(&mtc->mtProgress, t->index);
if (res != SZ_OK)
MtProgress_SetError(&mtc->mtProgress, res);
}
{
CMtCoderBlock *block = &mtc->blocks[bi];
block->res = res;
block->bufIndex = bufIndex;
block->finished = finished;
}
#ifdef MTCODER__USE_WRITE_THREAD
RINOK_THREAD(Event_Set(&mtc->writeEvents[bi]))
#else
{
unsigned wi;
{
CriticalSection_Enter(&mtc->cs);
wi = mtc->writeIndex;
if (wi == bi)
mtc->writeIndex = (unsigned)(int)-1;
else
mtc->ReadyBlocks[bi] = True;
CriticalSection_Leave(&mtc->cs);
}
if (wi != bi)
{
if (res != SZ_OK || finished)
return 0;
continue;
}
if (mtc->writeRes != SZ_OK)
res = mtc->writeRes;
for (;;)
{
if (res == SZ_OK && bufIndex != (unsigned)(int)-1)
{
res = mtc->mtCallback->Write(mtc->mtCallbackObject, bufIndex);
if (res != SZ_OK)
{
mtc->writeRes = res;
MtProgress_SetError(&mtc->mtProgress, res);
}
}
if (++wi >= mtc->numBlocksMax)
wi = 0;
{
BoolInt isReady;
CriticalSection_Enter(&mtc->cs);
if (bufIndex != (unsigned)(int)-1)
{
mtc->freeBlockList[bufIndex] = mtc->freeBlockHead;
mtc->freeBlockHead = bufIndex;
}
isReady = mtc->ReadyBlocks[wi];
if (isReady)
mtc->ReadyBlocks[wi] = False;
else
mtc->writeIndex = wi;
CriticalSection_Leave(&mtc->cs);
RINOK_THREAD(Semaphore_Release1(&mtc->blocksSemaphore))
if (!isReady)
break;
}
{
CMtCoderBlock *block = &mtc->blocks[wi];
if (res == SZ_OK && block->res != SZ_OK)
res = block->res;
bufIndex = block->bufIndex;
finished = block->finished;
}
}
}
#endif
if (finished || res != SZ_OK)
return 0;
}
}
static THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE ThreadFunc(void *pp)
{
CMtCoderThread *t = (CMtCoderThread *)pp;
for (;;)
{
if (Event_Wait(&t->startEvent) != 0)
return SZ_ERROR_THREAD;
if (t->stop)
return 0;
{
SRes res = ThreadFunc2(t);
CMtCoder *mtc = t->mtCoder;
if (res != SZ_OK)
{
MtProgress_SetError(&mtc->mtProgress, res);
}
#ifndef MTCODER__USE_WRITE_THREAD
{
unsigned numFinished = (unsigned)InterlockedIncrement(&mtc->numFinishedThreads);
if (numFinished == mtc->numStartedThreads)
if (Event_Set(&mtc->finishedEvent) != 0)
return SZ_ERROR_THREAD;
}
#endif
}
}
}
void MtCoder_Construct(CMtCoder *p)
{
unsigned i;
p->blockSize = 0;
p->numThreadsMax = 0;
p->expectedDataSize = (UInt64)(Int64)-1;
p->inStream = NULL;
p->inData = NULL;
p->inDataSize = 0;
p->progress = NULL;
p->allocBig = NULL;
p->mtCallback = NULL;
p->mtCallbackObject = NULL;
p->allocatedBufsSize = 0;
Event_Construct(&p->readEvent);
Semaphore_Construct(&p->blocksSemaphore);
for (i = 0; i < MTCODER__THREADS_MAX; i++)
{
CMtCoderThread *t = &p->threads[i];
t->mtCoder = p;
t->index = i;
t->inBuf = NULL;
t->stop = False;
Event_Construct(&t->startEvent);
Thread_Construct(&t->thread);
}
#ifdef MTCODER__USE_WRITE_THREAD
for (i = 0; i < MTCODER__BLOCKS_MAX; i++)
Event_Construct(&p->writeEvents[i]);
#else
Event_Construct(&p->finishedEvent);
#endif
CriticalSection_Init(&p->cs);
CriticalSection_Init(&p->mtProgress.cs);
}
static void MtCoder_Free(CMtCoder *p)
{
unsigned i;
/*
p->stopReading = True;
if (Event_IsCreated(&p->readEvent))
Event_Set(&p->readEvent);
*/
for (i = 0; i < MTCODER__THREADS_MAX; i++)
MtCoderThread_Destruct(&p->threads[i]);
Event_Close(&p->readEvent);
Semaphore_Close(&p->blocksSemaphore);
#ifdef MTCODER__USE_WRITE_THREAD
for (i = 0; i < MTCODER__BLOCKS_MAX; i++)
Event_Close(&p->writeEvents[i]);
#else
Event_Close(&p->finishedEvent);
#endif
}
void MtCoder_Destruct(CMtCoder *p)
{
MtCoder_Free(p);
CriticalSection_Delete(&p->cs);
CriticalSection_Delete(&p->mtProgress.cs);
}
SRes MtCoder_Code(CMtCoder *p)
{
unsigned numThreads = p->numThreadsMax;
unsigned numBlocksMax;
unsigned i;
SRes res = SZ_OK;
if (numThreads > MTCODER__THREADS_MAX)
numThreads = MTCODER__THREADS_MAX;
numBlocksMax = MTCODER__GET_NUM_BLOCKS_FROM_THREADS(numThreads);
if (p->blockSize < ((UInt32)1 << 26)) numBlocksMax++;
if (p->blockSize < ((UInt32)1 << 24)) numBlocksMax++;
if (p->blockSize < ((UInt32)1 << 22)) numBlocksMax++;
if (numBlocksMax > MTCODER__BLOCKS_MAX)
numBlocksMax = MTCODER__BLOCKS_MAX;
if (p->blockSize != p->allocatedBufsSize)
{
for (i = 0; i < MTCODER__THREADS_MAX; i++)
{
CMtCoderThread *t = &p->threads[i];
if (t->inBuf)
{
ISzAlloc_Free(p->allocBig, t->inBuf);
t->inBuf = NULL;
}
}
p->allocatedBufsSize = p->blockSize;
}
p->readRes = SZ_OK;
MtProgress_Init(&p->mtProgress, p->progress);
#ifdef MTCODER__USE_WRITE_THREAD
for (i = 0; i < numBlocksMax; i++)
{
RINOK_THREAD(ArEvent_OptCreate_And_Reset(&p->writeEvents[i]));
}
#else
RINOK_THREAD(ArEvent_OptCreate_And_Reset(&p->finishedEvent));
#endif
{
RINOK_THREAD(ArEvent_OptCreate_And_Reset(&p->readEvent));
if (Semaphore_IsCreated(&p->blocksSemaphore))
{
RINOK_THREAD(Semaphore_Close(&p->blocksSemaphore));
}
RINOK_THREAD(Semaphore_Create(&p->blocksSemaphore, numBlocksMax, numBlocksMax));
}
for (i = 0; i < MTCODER__BLOCKS_MAX - 1; i++)
p->freeBlockList[i] = i + 1;
p->freeBlockList[MTCODER__BLOCKS_MAX - 1] = (unsigned)(int)-1;
p->freeBlockHead = 0;
p->readProcessed = 0;
p->blockIndex = 0;
p->numBlocksMax = numBlocksMax;
p->stopReading = False;
#ifndef MTCODER__USE_WRITE_THREAD
p->writeIndex = 0;
p->writeRes = SZ_OK;
for (i = 0; i < MTCODER__BLOCKS_MAX; i++)
p->ReadyBlocks[i] = False;
p->numFinishedThreads = 0;
#endif
p->numStartedThreadsLimit = numThreads;
p->numStartedThreads = 0;
// for (i = 0; i < numThreads; i++)
{
CMtCoderThread *nextThread = &p->threads[p->numStartedThreads++];
RINOK(MtCoderThread_CreateAndStart(nextThread));
}
RINOK_THREAD(Event_Set(&p->readEvent))
#ifdef MTCODER__USE_WRITE_THREAD
{
unsigned bi = 0;
for (;; bi++)
{
if (bi >= numBlocksMax)
bi = 0;
RINOK_THREAD(Event_Wait(&p->writeEvents[bi]))
{
const CMtCoderBlock *block = &p->blocks[bi];
unsigned bufIndex = block->bufIndex;
BoolInt finished = block->finished;
if (res == SZ_OK && block->res != SZ_OK)
res = block->res;
if (bufIndex != (unsigned)(int)-1)
{
if (res == SZ_OK)
{
res = p->mtCallback->Write(p->mtCallbackObject, bufIndex);
if (res != SZ_OK)
MtProgress_SetError(&p->mtProgress, res);
}
CriticalSection_Enter(&p->cs);
{
p->freeBlockList[bufIndex] = p->freeBlockHead;
p->freeBlockHead = bufIndex;
}
CriticalSection_Leave(&p->cs);
}
RINOK_THREAD(Semaphore_Release1(&p->blocksSemaphore))
if (finished)
break;
}
}
}
#else
{
WRes wres = Event_Wait(&p->finishedEvent);
res = MY_SRes_HRESULT_FROM_WRes(wres);
}
#endif
if (res == SZ_OK)
res = p->readRes;
if (res == SZ_OK)
res = p->mtProgress.res;
#ifndef MTCODER__USE_WRITE_THREAD
if (res == SZ_OK)
res = p->writeRes;
#endif
if (res != SZ_OK)
MtCoder_Free(p);
return res;
}
#endif
+141
View File
@@ -0,0 +1,141 @@
/* MtCoder.h -- Multi-thread Coder
2018-07-04 : Igor Pavlov : Public domain */
#ifndef __MT_CODER_H
#define __MT_CODER_H
#include "MtDec.h"
EXTERN_C_BEGIN
/*
if ( defined MTCODER__USE_WRITE_THREAD) : main thread writes all data blocks to output stream
if (not defined MTCODER__USE_WRITE_THREAD) : any coder thread can write data blocks to output stream
*/
/* #define MTCODER__USE_WRITE_THREAD */
#ifndef _7ZIP_ST
#define MTCODER__GET_NUM_BLOCKS_FROM_THREADS(numThreads) ((numThreads) + (numThreads) / 8 + 1)
#define MTCODER__THREADS_MAX 64
#define MTCODER__BLOCKS_MAX (MTCODER__GET_NUM_BLOCKS_FROM_THREADS(MTCODER__THREADS_MAX) + 3)
#else
#define MTCODER__THREADS_MAX 1
#define MTCODER__BLOCKS_MAX 1
#endif
#ifndef _7ZIP_ST
typedef struct
{
ICompressProgress vt;
CMtProgress *mtProgress;
UInt64 inSize;
UInt64 outSize;
} CMtProgressThunk;
void MtProgressThunk_CreateVTable(CMtProgressThunk *p);
#define MtProgressThunk_Init(p) { (p)->inSize = 0; (p)->outSize = 0; }
struct _CMtCoder;
typedef struct
{
struct _CMtCoder *mtCoder;
unsigned index;
int stop;
Byte *inBuf;
CAutoResetEvent startEvent;
CThread thread;
} CMtCoderThread;
typedef struct
{
SRes (*Code)(void *p, unsigned coderIndex, unsigned outBufIndex,
const Byte *src, size_t srcSize, int finished);
SRes (*Write)(void *p, unsigned outBufIndex);
} IMtCoderCallback2;
typedef struct
{
SRes res;
unsigned bufIndex;
BoolInt finished;
} CMtCoderBlock;
typedef struct _CMtCoder
{
/* input variables */
size_t blockSize; /* size of input block */
unsigned numThreadsMax;
UInt64 expectedDataSize;
ISeqInStream *inStream;
const Byte *inData;
size_t inDataSize;
ICompressProgress *progress;
ISzAllocPtr allocBig;
IMtCoderCallback2 *mtCallback;
void *mtCallbackObject;
/* internal variables */
size_t allocatedBufsSize;
CAutoResetEvent readEvent;
CSemaphore blocksSemaphore;
BoolInt stopReading;
SRes readRes;
#ifdef MTCODER__USE_WRITE_THREAD
CAutoResetEvent writeEvents[MTCODER__BLOCKS_MAX];
#else
CAutoResetEvent finishedEvent;
SRes writeRes;
unsigned writeIndex;
Byte ReadyBlocks[MTCODER__BLOCKS_MAX];
LONG numFinishedThreads;
#endif
unsigned numStartedThreadsLimit;
unsigned numStartedThreads;
unsigned numBlocksMax;
unsigned blockIndex;
UInt64 readProcessed;
CCriticalSection cs;
unsigned freeBlockHead;
unsigned freeBlockList[MTCODER__BLOCKS_MAX];
CMtProgress mtProgress;
CMtCoderBlock blocks[MTCODER__BLOCKS_MAX];
CMtCoderThread threads[MTCODER__THREADS_MAX];
} CMtCoder;
void MtCoder_Construct(CMtCoder *p);
void MtCoder_Destruct(CMtCoder *p);
SRes MtCoder_Code(CMtCoder *p);
#endif
EXTERN_C_END
#endif
+1138
View File
File diff suppressed because it is too large Load Diff
+201
View File
@@ -0,0 +1,201 @@
/* MtDec.h -- Multi-thread Decoder
2018-07-04 : Igor Pavlov : Public domain */
#ifndef __MT_DEC_H
#define __MT_DEC_H
#include "7zTypes.h"
#ifndef _7ZIP_ST
#include "Threads.h"
#endif
EXTERN_C_BEGIN
#ifndef _7ZIP_ST
#ifndef _7ZIP_ST
#define MTDEC__THREADS_MAX 32
#else
#define MTDEC__THREADS_MAX 1
#endif
typedef struct
{
ICompressProgress *progress;
SRes res;
UInt64 totalInSize;
UInt64 totalOutSize;
CCriticalSection cs;
} CMtProgress;
void MtProgress_Init(CMtProgress *p, ICompressProgress *progress);
SRes MtProgress_Progress_ST(CMtProgress *p);
SRes MtProgress_ProgressAdd(CMtProgress *p, UInt64 inSize, UInt64 outSize);
SRes MtProgress_GetError(CMtProgress *p);
void MtProgress_SetError(CMtProgress *p, SRes res);
struct _CMtDec;
typedef struct
{
struct _CMtDec *mtDec;
unsigned index;
void *inBuf;
size_t inDataSize_Start; // size of input data in start block
UInt64 inDataSize; // total size of input data in all blocks
CThread thread;
CAutoResetEvent canRead;
CAutoResetEvent canWrite;
void *allocaPtr;
} CMtDecThread;
void MtDecThread_FreeInBufs(CMtDecThread *t);
typedef enum
{
MTDEC_PARSE_CONTINUE, // continue this block with more input data
MTDEC_PARSE_OVERFLOW, // MT buffers overflow, need switch to single-thread
MTDEC_PARSE_NEW, // new block
MTDEC_PARSE_END // end of block threading. But we still can return to threading after Write(&needContinue)
} EMtDecParseState;
typedef struct
{
// in
int startCall;
const Byte *src;
size_t srcSize;
// in : (srcSize == 0) is allowed
// out : it's allowed to return less that actually was used ?
int srcFinished;
// out
EMtDecParseState state;
BoolInt canCreateNewThread;
UInt64 outPos; // check it (size_t)
} CMtDecCallbackInfo;
typedef struct
{
void (*Parse)(void *p, unsigned coderIndex, CMtDecCallbackInfo *ci);
// PreCode() and Code():
// (SRes_return_result != SZ_OK) means stop decoding, no need another blocks
SRes (*PreCode)(void *p, unsigned coderIndex);
SRes (*Code)(void *p, unsigned coderIndex,
const Byte *src, size_t srcSize, int srcFinished,
UInt64 *inCodePos, UInt64 *outCodePos, int *stop);
// stop - means stop another Code calls
/* Write() must be called, if Parse() was called
set (needWrite) if
{
&& (was not interrupted by progress)
&& (was not interrupted in previous block)
}
out:
if (*needContinue), decoder still need to continue decoding with new iteration,
even after MTDEC_PARSE_END
if (*canRecode), we didn't flush current block data, so we still can decode current block later.
*/
SRes (*Write)(void *p, unsigned coderIndex,
BoolInt needWriteToStream,
const Byte *src, size_t srcSize,
// int srcFinished,
BoolInt *needContinue,
BoolInt *canRecode);
} IMtDecCallback;
typedef struct _CMtDec
{
/* input variables */
size_t inBufSize; /* size of input block */
unsigned numThreadsMax;
// size_t inBlockMax;
unsigned numThreadsMax_2;
ISeqInStream *inStream;
// const Byte *inData;
// size_t inDataSize;
ICompressProgress *progress;
ISzAllocPtr alloc;
IMtDecCallback *mtCallback;
void *mtCallbackObject;
/* internal variables */
size_t allocatedBufsSize;
BoolInt exitThread;
WRes exitThreadWRes;
UInt64 blockIndex;
BoolInt isAllocError;
BoolInt overflow;
SRes threadingErrorSRes;
BoolInt needContinue;
// CAutoResetEvent finishedEvent;
SRes readRes;
SRes codeRes;
BoolInt wasInterrupted;
unsigned numStartedThreads_Limit;
unsigned numStartedThreads;
Byte *crossBlock;
size_t crossStart;
size_t crossEnd;
UInt64 readProcessed;
BoolInt readWasFinished;
UInt64 inProcessed;
unsigned filledThreadStart;
unsigned numFilledThreads;
#ifndef _7ZIP_ST
BoolInt needInterrupt;
UInt64 interruptIndex;
CMtProgress mtProgress;
CMtDecThread threads[MTDEC__THREADS_MAX];
#endif
} CMtDec;
void MtDec_Construct(CMtDec *p);
void MtDec_Destruct(CMtDec *p);
/*
MtDec_Code() returns:
SZ_OK - in most cases
MY_SRes_HRESULT_FROM_WRes(WRes_error) - in case of unexpected error in threading function
*/
SRes MtDec_Code(CMtDec *p);
Byte *MtDec_GetCrossBuff(CMtDec *p);
int MtDec_PrepareRead(CMtDec *p);
const Byte *MtDec_Read(CMtDec *p, size_t *inLim);
#endif
EXTERN_C_END
#endif
+85
View File
@@ -0,0 +1,85 @@
/* Ppmd.h -- PPMD codec common code
2017-04-03 : Igor Pavlov : Public domain
This code is based on PPMd var.H (2001): Dmitry Shkarin : Public domain */
#ifndef __PPMD_H
#define __PPMD_H
#include "CpuArch.h"
EXTERN_C_BEGIN
#ifdef MY_CPU_32BIT
#define PPMD_32BIT
#endif
#define PPMD_INT_BITS 7
#define PPMD_PERIOD_BITS 7
#define PPMD_BIN_SCALE (1 << (PPMD_INT_BITS + PPMD_PERIOD_BITS))
#define PPMD_GET_MEAN_SPEC(summ, shift, round) (((summ) + (1 << ((shift) - (round)))) >> (shift))
#define PPMD_GET_MEAN(summ) PPMD_GET_MEAN_SPEC((summ), PPMD_PERIOD_BITS, 2)
#define PPMD_UPDATE_PROB_0(prob) ((prob) + (1 << PPMD_INT_BITS) - PPMD_GET_MEAN(prob))
#define PPMD_UPDATE_PROB_1(prob) ((prob) - PPMD_GET_MEAN(prob))
#define PPMD_N1 4
#define PPMD_N2 4
#define PPMD_N3 4
#define PPMD_N4 ((128 + 3 - 1 * PPMD_N1 - 2 * PPMD_N2 - 3 * PPMD_N3) / 4)
#define PPMD_NUM_INDEXES (PPMD_N1 + PPMD_N2 + PPMD_N3 + PPMD_N4)
#pragma pack(push, 1)
/* Most compilers works OK here even without #pragma pack(push, 1), but some GCC compilers need it. */
/* SEE-contexts for PPM-contexts with masked symbols */
typedef struct
{
UInt16 Summ; /* Freq */
Byte Shift; /* Speed of Freq change; low Shift is for fast change */
Byte Count; /* Count to next change of Shift */
} CPpmd_See;
#define Ppmd_See_Update(p) if ((p)->Shift < PPMD_PERIOD_BITS && --(p)->Count == 0) \
{ (p)->Summ <<= 1; (p)->Count = (Byte)(3 << (p)->Shift++); }
typedef struct
{
Byte Symbol;
Byte Freq;
UInt16 SuccessorLow;
UInt16 SuccessorHigh;
} CPpmd_State;
#pragma pack(pop)
typedef
#ifdef PPMD_32BIT
CPpmd_State *
#else
UInt32
#endif
CPpmd_State_Ref;
typedef
#ifdef PPMD_32BIT
void *
#else
UInt32
#endif
CPpmd_Void_Ref;
typedef
#ifdef PPMD_32BIT
Byte *
#else
UInt32
#endif
CPpmd_Byte_Ref;
#define PPMD_SetAllBitsIn256Bytes(p) \
{ size_t z; for (z = 0; z < 256 / sizeof(p[0]); z += 8) { \
p[z+7] = p[z+6] = p[z+5] = p[z+4] = p[z+3] = p[z+2] = p[z+1] = p[z+0] = ~(size_t)0; }}
EXTERN_C_END
#endif
+712
View File
@@ -0,0 +1,712 @@
/* Ppmd7.c -- PPMdH codec
2018-07-04 : Igor Pavlov : Public domain
This code is based on PPMd var.H (2001): Dmitry Shkarin : Public domain */
#include "Precomp.h"
#include <string.h>
#include "Ppmd7.h"
const Byte PPMD7_kExpEscape[16] = { 25, 14, 9, 7, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 2 };
static const UInt16 kInitBinEsc[] = { 0x3CDD, 0x1F3F, 0x59BF, 0x48F3, 0x64A1, 0x5ABC, 0x6632, 0x6051};
#define MAX_FREQ 124
#define UNIT_SIZE 12
#define U2B(nu) ((UInt32)(nu) * UNIT_SIZE)
#define U2I(nu) (p->Units2Indx[(size_t)(nu) - 1])
#define I2U(indx) (p->Indx2Units[indx])
#ifdef PPMD_32BIT
#define REF(ptr) (ptr)
#else
#define REF(ptr) ((UInt32)((Byte *)(ptr) - (p)->Base))
#endif
#define STATS_REF(ptr) ((CPpmd_State_Ref)REF(ptr))
#define CTX(ref) ((CPpmd7_Context *)Ppmd7_GetContext(p, ref))
#define STATS(ctx) Ppmd7_GetStats(p, ctx)
#define ONE_STATE(ctx) Ppmd7Context_OneState(ctx)
#define SUFFIX(ctx) CTX((ctx)->Suffix)
typedef CPpmd7_Context * CTX_PTR;
struct CPpmd7_Node_;
typedef
#ifdef PPMD_32BIT
struct CPpmd7_Node_ *
#else
UInt32
#endif
CPpmd7_Node_Ref;
typedef struct CPpmd7_Node_
{
UInt16 Stamp; /* must be at offset 0 as CPpmd7_Context::NumStats. Stamp=0 means free */
UInt16 NU;
CPpmd7_Node_Ref Next; /* must be at offset >= 4 */
CPpmd7_Node_Ref Prev;
} CPpmd7_Node;
#ifdef PPMD_32BIT
#define NODE(ptr) (ptr)
#else
#define NODE(offs) ((CPpmd7_Node *)(p->Base + (offs)))
#endif
void Ppmd7_Construct(CPpmd7 *p)
{
unsigned i, k, m;
p->Base = 0;
for (i = 0, k = 0; i < PPMD_NUM_INDEXES; i++)
{
unsigned step = (i >= 12 ? 4 : (i >> 2) + 1);
do { p->Units2Indx[k++] = (Byte)i; } while (--step);
p->Indx2Units[i] = (Byte)k;
}
p->NS2BSIndx[0] = (0 << 1);
p->NS2BSIndx[1] = (1 << 1);
memset(p->NS2BSIndx + 2, (2 << 1), 9);
memset(p->NS2BSIndx + 11, (3 << 1), 256 - 11);
for (i = 0; i < 3; i++)
p->NS2Indx[i] = (Byte)i;
for (m = i, k = 1; i < 256; i++)
{
p->NS2Indx[i] = (Byte)m;
if (--k == 0)
k = (++m) - 2;
}
memset(p->HB2Flag, 0, 0x40);
memset(p->HB2Flag + 0x40, 8, 0x100 - 0x40);
}
void Ppmd7_Free(CPpmd7 *p, ISzAllocPtr alloc)
{
ISzAlloc_Free(alloc, p->Base);
p->Size = 0;
p->Base = 0;
}
BoolInt Ppmd7_Alloc(CPpmd7 *p, UInt32 size, ISzAllocPtr alloc)
{
if (!p->Base || p->Size != size)
{
size_t size2;
Ppmd7_Free(p, alloc);
size2 = 0
#ifndef PPMD_32BIT
+ UNIT_SIZE
#endif
;
p->AlignOffset =
#ifdef PPMD_32BIT
(4 - size) & 3;
#else
4 - (size & 3);
#endif
if ((p->Base = (Byte *)ISzAlloc_Alloc(alloc, p->AlignOffset + size + size2)) == 0)
return False;
p->Size = size;
}
return True;
}
static void InsertNode(CPpmd7 *p, void *node, unsigned indx)
{
*((CPpmd_Void_Ref *)node) = p->FreeList[indx];
p->FreeList[indx] = REF(node);
}
static void *RemoveNode(CPpmd7 *p, unsigned indx)
{
CPpmd_Void_Ref *node = (CPpmd_Void_Ref *)Ppmd7_GetPtr(p, p->FreeList[indx]);
p->FreeList[indx] = *node;
return node;
}
static void SplitBlock(CPpmd7 *p, void *ptr, unsigned oldIndx, unsigned newIndx)
{
unsigned i, nu = I2U(oldIndx) - I2U(newIndx);
ptr = (Byte *)ptr + U2B(I2U(newIndx));
if (I2U(i = U2I(nu)) != nu)
{
unsigned k = I2U(--i);
InsertNode(p, ((Byte *)ptr) + U2B(k), nu - k - 1);
}
InsertNode(p, ptr, i);
}
static void GlueFreeBlocks(CPpmd7 *p)
{
#ifdef PPMD_32BIT
CPpmd7_Node headItem;
CPpmd7_Node_Ref head = &headItem;
#else
CPpmd7_Node_Ref head = p->AlignOffset + p->Size;
#endif
CPpmd7_Node_Ref n = head;
unsigned i;
p->GlueCount = 255;
/* create doubly-linked list of free blocks */
for (i = 0; i < PPMD_NUM_INDEXES; i++)
{
UInt16 nu = I2U(i);
CPpmd7_Node_Ref next = (CPpmd7_Node_Ref)p->FreeList[i];
p->FreeList[i] = 0;
while (next != 0)
{
CPpmd7_Node *node = NODE(next);
node->Next = n;
n = NODE(n)->Prev = next;
next = *(const CPpmd7_Node_Ref *)node;
node->Stamp = 0;
node->NU = (UInt16)nu;
}
}
NODE(head)->Stamp = 1;
NODE(head)->Next = n;
NODE(n)->Prev = head;
if (p->LoUnit != p->HiUnit)
((CPpmd7_Node *)p->LoUnit)->Stamp = 1;
/* Glue free blocks */
while (n != head)
{
CPpmd7_Node *node = NODE(n);
UInt32 nu = (UInt32)node->NU;
for (;;)
{
CPpmd7_Node *node2 = NODE(n) + nu;
nu += node2->NU;
if (node2->Stamp != 0 || nu >= 0x10000)
break;
NODE(node2->Prev)->Next = node2->Next;
NODE(node2->Next)->Prev = node2->Prev;
node->NU = (UInt16)nu;
}
n = node->Next;
}
/* Fill lists of free blocks */
for (n = NODE(head)->Next; n != head;)
{
CPpmd7_Node *node = NODE(n);
unsigned nu;
CPpmd7_Node_Ref next = node->Next;
for (nu = node->NU; nu > 128; nu -= 128, node += 128)
InsertNode(p, node, PPMD_NUM_INDEXES - 1);
if (I2U(i = U2I(nu)) != nu)
{
unsigned k = I2U(--i);
InsertNode(p, node + k, nu - k - 1);
}
InsertNode(p, node, i);
n = next;
}
}
static void *AllocUnitsRare(CPpmd7 *p, unsigned indx)
{
unsigned i;
void *retVal;
if (p->GlueCount == 0)
{
GlueFreeBlocks(p);
if (p->FreeList[indx] != 0)
return RemoveNode(p, indx);
}
i = indx;
do
{
if (++i == PPMD_NUM_INDEXES)
{
UInt32 numBytes = U2B(I2U(indx));
p->GlueCount--;
return ((UInt32)(p->UnitsStart - p->Text) > numBytes) ? (p->UnitsStart -= numBytes) : (NULL);
}
}
while (p->FreeList[i] == 0);
retVal = RemoveNode(p, i);
SplitBlock(p, retVal, i, indx);
return retVal;
}
static void *AllocUnits(CPpmd7 *p, unsigned indx)
{
UInt32 numBytes;
if (p->FreeList[indx] != 0)
return RemoveNode(p, indx);
numBytes = U2B(I2U(indx));
if (numBytes <= (UInt32)(p->HiUnit - p->LoUnit))
{
void *retVal = p->LoUnit;
p->LoUnit += numBytes;
return retVal;
}
return AllocUnitsRare(p, indx);
}
#define MyMem12Cpy(dest, src, num) \
{ UInt32 *d = (UInt32 *)dest; const UInt32 *s = (const UInt32 *)src; UInt32 n = num; \
do { d[0] = s[0]; d[1] = s[1]; d[2] = s[2]; s += 3; d += 3; } while (--n); }
static void *ShrinkUnits(CPpmd7 *p, void *oldPtr, unsigned oldNU, unsigned newNU)
{
unsigned i0 = U2I(oldNU);
unsigned i1 = U2I(newNU);
if (i0 == i1)
return oldPtr;
if (p->FreeList[i1] != 0)
{
void *ptr = RemoveNode(p, i1);
MyMem12Cpy(ptr, oldPtr, newNU);
InsertNode(p, oldPtr, i0);
return ptr;
}
SplitBlock(p, oldPtr, i0, i1);
return oldPtr;
}
#define SUCCESSOR(p) ((CPpmd_Void_Ref)((p)->SuccessorLow | ((UInt32)(p)->SuccessorHigh << 16)))
static void SetSuccessor(CPpmd_State *p, CPpmd_Void_Ref v)
{
(p)->SuccessorLow = (UInt16)((UInt32)(v) & 0xFFFF);
(p)->SuccessorHigh = (UInt16)(((UInt32)(v) >> 16) & 0xFFFF);
}
static void RestartModel(CPpmd7 *p)
{
unsigned i, k, m;
memset(p->FreeList, 0, sizeof(p->FreeList));
p->Text = p->Base + p->AlignOffset;
p->HiUnit = p->Text + p->Size;
p->LoUnit = p->UnitsStart = p->HiUnit - p->Size / 8 / UNIT_SIZE * 7 * UNIT_SIZE;
p->GlueCount = 0;
p->OrderFall = p->MaxOrder;
p->RunLength = p->InitRL = -(Int32)((p->MaxOrder < 12) ? p->MaxOrder : 12) - 1;
p->PrevSuccess = 0;
p->MinContext = p->MaxContext = (CTX_PTR)(p->HiUnit -= UNIT_SIZE); /* AllocContext(p); */
p->MinContext->Suffix = 0;
p->MinContext->NumStats = 256;
p->MinContext->SummFreq = 256 + 1;
p->FoundState = (CPpmd_State *)p->LoUnit; /* AllocUnits(p, PPMD_NUM_INDEXES - 1); */
p->LoUnit += U2B(256 / 2);
p->MinContext->Stats = REF(p->FoundState);
for (i = 0; i < 256; i++)
{
CPpmd_State *s = &p->FoundState[i];
s->Symbol = (Byte)i;
s->Freq = 1;
SetSuccessor(s, 0);
}
for (i = 0; i < 128; i++)
for (k = 0; k < 8; k++)
{
UInt16 *dest = p->BinSumm[i] + k;
UInt16 val = (UInt16)(PPMD_BIN_SCALE - kInitBinEsc[k] / (i + 2));
for (m = 0; m < 64; m += 8)
dest[m] = val;
}
for (i = 0; i < 25; i++)
for (k = 0; k < 16; k++)
{
CPpmd_See *s = &p->See[i][k];
s->Summ = (UInt16)((5 * i + 10) << (s->Shift = PPMD_PERIOD_BITS - 4));
s->Count = 4;
}
}
void Ppmd7_Init(CPpmd7 *p, unsigned maxOrder)
{
p->MaxOrder = maxOrder;
RestartModel(p);
p->DummySee.Shift = PPMD_PERIOD_BITS;
p->DummySee.Summ = 0; /* unused */
p->DummySee.Count = 64; /* unused */
}
static CTX_PTR CreateSuccessors(CPpmd7 *p, BoolInt skip)
{
CPpmd_State upState;
CTX_PTR c = p->MinContext;
CPpmd_Byte_Ref upBranch = (CPpmd_Byte_Ref)SUCCESSOR(p->FoundState);
CPpmd_State *ps[PPMD7_MAX_ORDER];
unsigned numPs = 0;
if (!skip)
ps[numPs++] = p->FoundState;
while (c->Suffix)
{
CPpmd_Void_Ref successor;
CPpmd_State *s;
c = SUFFIX(c);
if (c->NumStats != 1)
{
for (s = STATS(c); s->Symbol != p->FoundState->Symbol; s++);
}
else
s = ONE_STATE(c);
successor = SUCCESSOR(s);
if (successor != upBranch)
{
c = CTX(successor);
if (numPs == 0)
return c;
break;
}
ps[numPs++] = s;
}
upState.Symbol = *(const Byte *)Ppmd7_GetPtr(p, upBranch);
SetSuccessor(&upState, upBranch + 1);
if (c->NumStats == 1)
upState.Freq = ONE_STATE(c)->Freq;
else
{
UInt32 cf, s0;
CPpmd_State *s;
for (s = STATS(c); s->Symbol != upState.Symbol; s++);
cf = s->Freq - 1;
s0 = c->SummFreq - c->NumStats - cf;
upState.Freq = (Byte)(1 + ((2 * cf <= s0) ? (5 * cf > s0) : ((2 * cf + 3 * s0 - 1) / (2 * s0))));
}
do
{
/* Create Child */
CTX_PTR c1; /* = AllocContext(p); */
if (p->HiUnit != p->LoUnit)
c1 = (CTX_PTR)(p->HiUnit -= UNIT_SIZE);
else if (p->FreeList[0] != 0)
c1 = (CTX_PTR)RemoveNode(p, 0);
else
{
c1 = (CTX_PTR)AllocUnitsRare(p, 0);
if (!c1)
return NULL;
}
c1->NumStats = 1;
*ONE_STATE(c1) = upState;
c1->Suffix = REF(c);
SetSuccessor(ps[--numPs], REF(c1));
c = c1;
}
while (numPs != 0);
return c;
}
static void SwapStates(CPpmd_State *t1, CPpmd_State *t2)
{
CPpmd_State tmp = *t1;
*t1 = *t2;
*t2 = tmp;
}
static void UpdateModel(CPpmd7 *p)
{
CPpmd_Void_Ref successor, fSuccessor = SUCCESSOR(p->FoundState);
CTX_PTR c;
unsigned s0, ns;
if (p->FoundState->Freq < MAX_FREQ / 4 && p->MinContext->Suffix != 0)
{
c = SUFFIX(p->MinContext);
if (c->NumStats == 1)
{
CPpmd_State *s = ONE_STATE(c);
if (s->Freq < 32)
s->Freq++;
}
else
{
CPpmd_State *s = STATS(c);
if (s->Symbol != p->FoundState->Symbol)
{
do { s++; } while (s->Symbol != p->FoundState->Symbol);
if (s[0].Freq >= s[-1].Freq)
{
SwapStates(&s[0], &s[-1]);
s--;
}
}
if (s->Freq < MAX_FREQ - 9)
{
s->Freq += 2;
c->SummFreq += 2;
}
}
}
if (p->OrderFall == 0)
{
p->MinContext = p->MaxContext = CreateSuccessors(p, True);
if (p->MinContext == 0)
{
RestartModel(p);
return;
}
SetSuccessor(p->FoundState, REF(p->MinContext));
return;
}
*p->Text++ = p->FoundState->Symbol;
successor = REF(p->Text);
if (p->Text >= p->UnitsStart)
{
RestartModel(p);
return;
}
if (fSuccessor)
{
if (fSuccessor <= successor)
{
CTX_PTR cs = CreateSuccessors(p, False);
if (cs == NULL)
{
RestartModel(p);
return;
}
fSuccessor = REF(cs);
}
if (--p->OrderFall == 0)
{
successor = fSuccessor;
p->Text -= (p->MaxContext != p->MinContext);
}
}
else
{
SetSuccessor(p->FoundState, successor);
fSuccessor = REF(p->MinContext);
}
s0 = p->MinContext->SummFreq - (ns = p->MinContext->NumStats) - (p->FoundState->Freq - 1);
for (c = p->MaxContext; c != p->MinContext; c = SUFFIX(c))
{
unsigned ns1;
UInt32 cf, sf;
if ((ns1 = c->NumStats) != 1)
{
if ((ns1 & 1) == 0)
{
/* Expand for one UNIT */
unsigned oldNU = ns1 >> 1;
unsigned i = U2I(oldNU);
if (i != U2I((size_t)oldNU + 1))
{
void *ptr = AllocUnits(p, i + 1);
void *oldPtr;
if (!ptr)
{
RestartModel(p);
return;
}
oldPtr = STATS(c);
MyMem12Cpy(ptr, oldPtr, oldNU);
InsertNode(p, oldPtr, i);
c->Stats = STATS_REF(ptr);
}
}
c->SummFreq = (UInt16)(c->SummFreq + (2 * ns1 < ns) + 2 * ((4 * ns1 <= ns) & (c->SummFreq <= 8 * ns1)));
}
else
{
CPpmd_State *s = (CPpmd_State*)AllocUnits(p, 0);
if (!s)
{
RestartModel(p);
return;
}
*s = *ONE_STATE(c);
c->Stats = REF(s);
if (s->Freq < MAX_FREQ / 4 - 1)
s->Freq <<= 1;
else
s->Freq = MAX_FREQ - 4;
c->SummFreq = (UInt16)(s->Freq + p->InitEsc + (ns > 3));
}
cf = 2 * (UInt32)p->FoundState->Freq * (c->SummFreq + 6);
sf = (UInt32)s0 + c->SummFreq;
if (cf < 6 * sf)
{
cf = 1 + (cf > sf) + (cf >= 4 * sf);
c->SummFreq += 3;
}
else
{
cf = 4 + (cf >= 9 * sf) + (cf >= 12 * sf) + (cf >= 15 * sf);
c->SummFreq = (UInt16)(c->SummFreq + cf);
}
{
CPpmd_State *s = STATS(c) + ns1;
SetSuccessor(s, successor);
s->Symbol = p->FoundState->Symbol;
s->Freq = (Byte)cf;
c->NumStats = (UInt16)(ns1 + 1);
}
}
p->MaxContext = p->MinContext = CTX(fSuccessor);
}
static void Rescale(CPpmd7 *p)
{
unsigned i, adder, sumFreq, escFreq;
CPpmd_State *stats = STATS(p->MinContext);
CPpmd_State *s = p->FoundState;
{
CPpmd_State tmp = *s;
for (; s != stats; s--)
s[0] = s[-1];
*s = tmp;
}
escFreq = p->MinContext->SummFreq - s->Freq;
s->Freq += 4;
adder = (p->OrderFall != 0);
s->Freq = (Byte)((s->Freq + adder) >> 1);
sumFreq = s->Freq;
i = p->MinContext->NumStats - 1;
do
{
escFreq -= (++s)->Freq;
s->Freq = (Byte)((s->Freq + adder) >> 1);
sumFreq += s->Freq;
if (s[0].Freq > s[-1].Freq)
{
CPpmd_State *s1 = s;
CPpmd_State tmp = *s1;
do
s1[0] = s1[-1];
while (--s1 != stats && tmp.Freq > s1[-1].Freq);
*s1 = tmp;
}
}
while (--i);
if (s->Freq == 0)
{
unsigned numStats = p->MinContext->NumStats;
unsigned n0, n1;
do { i++; } while ((--s)->Freq == 0);
escFreq += i;
p->MinContext->NumStats = (UInt16)(p->MinContext->NumStats - i);
if (p->MinContext->NumStats == 1)
{
CPpmd_State tmp = *stats;
do
{
tmp.Freq = (Byte)(tmp.Freq - (tmp.Freq >> 1));
escFreq >>= 1;
}
while (escFreq > 1);
InsertNode(p, stats, U2I(((numStats + 1) >> 1)));
*(p->FoundState = ONE_STATE(p->MinContext)) = tmp;
return;
}
n0 = (numStats + 1) >> 1;
n1 = (p->MinContext->NumStats + 1) >> 1;
if (n0 != n1)
p->MinContext->Stats = STATS_REF(ShrinkUnits(p, stats, n0, n1));
}
p->MinContext->SummFreq = (UInt16)(sumFreq + escFreq - (escFreq >> 1));
p->FoundState = STATS(p->MinContext);
}
CPpmd_See *Ppmd7_MakeEscFreq(CPpmd7 *p, unsigned numMasked, UInt32 *escFreq)
{
CPpmd_See *see;
unsigned nonMasked = p->MinContext->NumStats - numMasked;
if (p->MinContext->NumStats != 256)
{
see = p->See[(unsigned)p->NS2Indx[(size_t)nonMasked - 1]] +
(nonMasked < (unsigned)SUFFIX(p->MinContext)->NumStats - p->MinContext->NumStats) +
2 * (unsigned)(p->MinContext->SummFreq < 11 * p->MinContext->NumStats) +
4 * (unsigned)(numMasked > nonMasked) +
p->HiBitsFlag;
{
unsigned r = (see->Summ >> see->Shift);
see->Summ = (UInt16)(see->Summ - r);
*escFreq = r + (r == 0);
}
}
else
{
see = &p->DummySee;
*escFreq = 1;
}
return see;
}
static void NextContext(CPpmd7 *p)
{
CTX_PTR c = CTX(SUCCESSOR(p->FoundState));
if (p->OrderFall == 0 && (Byte *)c > p->Text)
p->MinContext = p->MaxContext = c;
else
UpdateModel(p);
}
void Ppmd7_Update1(CPpmd7 *p)
{
CPpmd_State *s = p->FoundState;
s->Freq += 4;
p->MinContext->SummFreq += 4;
if (s[0].Freq > s[-1].Freq)
{
SwapStates(&s[0], &s[-1]);
p->FoundState = --s;
if (s->Freq > MAX_FREQ)
Rescale(p);
}
NextContext(p);
}
void Ppmd7_Update1_0(CPpmd7 *p)
{
p->PrevSuccess = (2 * p->FoundState->Freq > p->MinContext->SummFreq);
p->RunLength += p->PrevSuccess;
p->MinContext->SummFreq += 4;
if ((p->FoundState->Freq += 4) > MAX_FREQ)
Rescale(p);
NextContext(p);
}
void Ppmd7_UpdateBin(CPpmd7 *p)
{
p->FoundState->Freq = (Byte)(p->FoundState->Freq + (p->FoundState->Freq < 128 ? 1: 0));
p->PrevSuccess = 1;
p->RunLength++;
NextContext(p);
}
void Ppmd7_Update2(CPpmd7 *p)
{
p->MinContext->SummFreq += 4;
if ((p->FoundState->Freq += 4) > MAX_FREQ)
Rescale(p);
p->RunLength = p->InitRL;
UpdateModel(p);
}
+142
View File
@@ -0,0 +1,142 @@
/* Ppmd7.h -- PPMdH compression codec
2018-07-04 : Igor Pavlov : Public domain
This code is based on PPMd var.H (2001): Dmitry Shkarin : Public domain */
/* This code supports virtual RangeDecoder and includes the implementation
of RangeCoder from 7z, instead of RangeCoder from original PPMd var.H.
If you need the compatibility with original PPMd var.H, you can use external RangeDecoder */
#ifndef __PPMD7_H
#define __PPMD7_H
#include "Ppmd.h"
EXTERN_C_BEGIN
#define PPMD7_MIN_ORDER 2
#define PPMD7_MAX_ORDER 64
#define PPMD7_MIN_MEM_SIZE (1 << 11)
#define PPMD7_MAX_MEM_SIZE (0xFFFFFFFF - 12 * 3)
struct CPpmd7_Context_;
typedef
#ifdef PPMD_32BIT
struct CPpmd7_Context_ *
#else
UInt32
#endif
CPpmd7_Context_Ref;
typedef struct CPpmd7_Context_
{
UInt16 NumStats;
UInt16 SummFreq;
CPpmd_State_Ref Stats;
CPpmd7_Context_Ref Suffix;
} CPpmd7_Context;
#define Ppmd7Context_OneState(p) ((CPpmd_State *)&(p)->SummFreq)
typedef struct
{
CPpmd7_Context *MinContext, *MaxContext;
CPpmd_State *FoundState;
unsigned OrderFall, InitEsc, PrevSuccess, MaxOrder, HiBitsFlag;
Int32 RunLength, InitRL; /* must be 32-bit at least */
UInt32 Size;
UInt32 GlueCount;
Byte *Base, *LoUnit, *HiUnit, *Text, *UnitsStart;
UInt32 AlignOffset;
Byte Indx2Units[PPMD_NUM_INDEXES];
Byte Units2Indx[128];
CPpmd_Void_Ref FreeList[PPMD_NUM_INDEXES];
Byte NS2Indx[256], NS2BSIndx[256], HB2Flag[256];
CPpmd_See DummySee, See[25][16];
UInt16 BinSumm[128][64];
} CPpmd7;
void Ppmd7_Construct(CPpmd7 *p);
BoolInt Ppmd7_Alloc(CPpmd7 *p, UInt32 size, ISzAllocPtr alloc);
void Ppmd7_Free(CPpmd7 *p, ISzAllocPtr alloc);
void Ppmd7_Init(CPpmd7 *p, unsigned maxOrder);
#define Ppmd7_WasAllocated(p) ((p)->Base != NULL)
/* ---------- Internal Functions ---------- */
extern const Byte PPMD7_kExpEscape[16];
#ifdef PPMD_32BIT
#define Ppmd7_GetPtr(p, ptr) (ptr)
#define Ppmd7_GetContext(p, ptr) (ptr)
#define Ppmd7_GetStats(p, ctx) ((ctx)->Stats)
#else
#define Ppmd7_GetPtr(p, offs) ((void *)((p)->Base + (offs)))
#define Ppmd7_GetContext(p, offs) ((CPpmd7_Context *)Ppmd7_GetPtr((p), (offs)))
#define Ppmd7_GetStats(p, ctx) ((CPpmd_State *)Ppmd7_GetPtr((p), ((ctx)->Stats)))
#endif
void Ppmd7_Update1(CPpmd7 *p);
void Ppmd7_Update1_0(CPpmd7 *p);
void Ppmd7_Update2(CPpmd7 *p);
void Ppmd7_UpdateBin(CPpmd7 *p);
#define Ppmd7_GetBinSumm(p) \
&p->BinSumm[(size_t)(unsigned)Ppmd7Context_OneState(p->MinContext)->Freq - 1][p->PrevSuccess + \
p->NS2BSIndx[(size_t)Ppmd7_GetContext(p, p->MinContext->Suffix)->NumStats - 1] + \
(p->HiBitsFlag = p->HB2Flag[p->FoundState->Symbol]) + \
2 * p->HB2Flag[(unsigned)Ppmd7Context_OneState(p->MinContext)->Symbol] + \
((p->RunLength >> 26) & 0x20)]
CPpmd_See *Ppmd7_MakeEscFreq(CPpmd7 *p, unsigned numMasked, UInt32 *scale);
/* ---------- Decode ---------- */
typedef struct IPpmd7_RangeDec IPpmd7_RangeDec;
struct IPpmd7_RangeDec
{
UInt32 (*GetThreshold)(const IPpmd7_RangeDec *p, UInt32 total);
void (*Decode)(const IPpmd7_RangeDec *p, UInt32 start, UInt32 size);
UInt32 (*DecodeBit)(const IPpmd7_RangeDec *p, UInt32 size0);
};
typedef struct
{
IPpmd7_RangeDec vt;
UInt32 Range;
UInt32 Code;
IByteIn *Stream;
} CPpmd7z_RangeDec;
void Ppmd7z_RangeDec_CreateVTable(CPpmd7z_RangeDec *p);
BoolInt Ppmd7z_RangeDec_Init(CPpmd7z_RangeDec *p);
#define Ppmd7z_RangeDec_IsFinishedOK(p) ((p)->Code == 0)
int Ppmd7_DecodeSymbol(CPpmd7 *p, const IPpmd7_RangeDec *rc);
/* ---------- Encode ---------- */
typedef struct
{
UInt64 Low;
UInt32 Range;
Byte Cache;
UInt64 CacheSize;
IByteOut *Stream;
} CPpmd7z_RangeEnc;
void Ppmd7z_RangeEnc_Init(CPpmd7z_RangeEnc *p);
void Ppmd7z_RangeEnc_FlushData(CPpmd7z_RangeEnc *p);
void Ppmd7_EncodeSymbol(CPpmd7 *p, CPpmd7z_RangeEnc *rc, int symbol);
EXTERN_C_END
#endif
+191
View File
@@ -0,0 +1,191 @@
/* Ppmd7Dec.c -- PPMdH Decoder
2018-07-04 : Igor Pavlov : Public domain
This code is based on PPMd var.H (2001): Dmitry Shkarin : Public domain */
#include "Precomp.h"
#include "Ppmd7.h"
#define kTopValue (1 << 24)
BoolInt Ppmd7z_RangeDec_Init(CPpmd7z_RangeDec *p)
{
unsigned i;
p->Code = 0;
p->Range = 0xFFFFFFFF;
if (IByteIn_Read(p->Stream) != 0)
return False;
for (i = 0; i < 4; i++)
p->Code = (p->Code << 8) | IByteIn_Read(p->Stream);
return (p->Code < 0xFFFFFFFF);
}
#define GET_Ppmd7z_RangeDec CPpmd7z_RangeDec *p = CONTAINER_FROM_VTBL(pp, CPpmd7z_RangeDec, vt);
static UInt32 Range_GetThreshold(const IPpmd7_RangeDec *pp, UInt32 total)
{
GET_Ppmd7z_RangeDec
return p->Code / (p->Range /= total);
}
static void Range_Normalize(CPpmd7z_RangeDec *p)
{
if (p->Range < kTopValue)
{
p->Code = (p->Code << 8) | IByteIn_Read(p->Stream);
p->Range <<= 8;
if (p->Range < kTopValue)
{
p->Code = (p->Code << 8) | IByteIn_Read(p->Stream);
p->Range <<= 8;
}
}
}
static void Range_Decode(const IPpmd7_RangeDec *pp, UInt32 start, UInt32 size)
{
GET_Ppmd7z_RangeDec
p->Code -= start * p->Range;
p->Range *= size;
Range_Normalize(p);
}
static UInt32 Range_DecodeBit(const IPpmd7_RangeDec *pp, UInt32 size0)
{
GET_Ppmd7z_RangeDec
UInt32 newBound = (p->Range >> 14) * size0;
UInt32 symbol;
if (p->Code < newBound)
{
symbol = 0;
p->Range = newBound;
}
else
{
symbol = 1;
p->Code -= newBound;
p->Range -= newBound;
}
Range_Normalize(p);
return symbol;
}
void Ppmd7z_RangeDec_CreateVTable(CPpmd7z_RangeDec *p)
{
p->vt.GetThreshold = Range_GetThreshold;
p->vt.Decode = Range_Decode;
p->vt.DecodeBit = Range_DecodeBit;
}
#define MASK(sym) ((signed char *)charMask)[sym]
int Ppmd7_DecodeSymbol(CPpmd7 *p, const IPpmd7_RangeDec *rc)
{
size_t charMask[256 / sizeof(size_t)];
if (p->MinContext->NumStats != 1)
{
CPpmd_State *s = Ppmd7_GetStats(p, p->MinContext);
unsigned i;
UInt32 count, hiCnt;
if ((count = rc->GetThreshold(rc, p->MinContext->SummFreq)) < (hiCnt = s->Freq))
{
Byte symbol;
rc->Decode(rc, 0, s->Freq);
p->FoundState = s;
symbol = s->Symbol;
Ppmd7_Update1_0(p);
return symbol;
}
p->PrevSuccess = 0;
i = p->MinContext->NumStats - 1;
do
{
if ((hiCnt += (++s)->Freq) > count)
{
Byte symbol;
rc->Decode(rc, hiCnt - s->Freq, s->Freq);
p->FoundState = s;
symbol = s->Symbol;
Ppmd7_Update1(p);
return symbol;
}
}
while (--i);
if (count >= p->MinContext->SummFreq)
return -2;
p->HiBitsFlag = p->HB2Flag[p->FoundState->Symbol];
rc->Decode(rc, hiCnt, p->MinContext->SummFreq - hiCnt);
PPMD_SetAllBitsIn256Bytes(charMask);
MASK(s->Symbol) = 0;
i = p->MinContext->NumStats - 1;
do { MASK((--s)->Symbol) = 0; } while (--i);
}
else
{
UInt16 *prob = Ppmd7_GetBinSumm(p);
if (rc->DecodeBit(rc, *prob) == 0)
{
Byte symbol;
*prob = (UInt16)PPMD_UPDATE_PROB_0(*prob);
symbol = (p->FoundState = Ppmd7Context_OneState(p->MinContext))->Symbol;
Ppmd7_UpdateBin(p);
return symbol;
}
*prob = (UInt16)PPMD_UPDATE_PROB_1(*prob);
p->InitEsc = PPMD7_kExpEscape[*prob >> 10];
PPMD_SetAllBitsIn256Bytes(charMask);
MASK(Ppmd7Context_OneState(p->MinContext)->Symbol) = 0;
p->PrevSuccess = 0;
}
for (;;)
{
CPpmd_State *ps[256], *s;
UInt32 freqSum, count, hiCnt;
CPpmd_See *see;
unsigned i, num, numMasked = p->MinContext->NumStats;
do
{
p->OrderFall++;
if (!p->MinContext->Suffix)
return -1;
p->MinContext = Ppmd7_GetContext(p, p->MinContext->Suffix);
}
while (p->MinContext->NumStats == numMasked);
hiCnt = 0;
s = Ppmd7_GetStats(p, p->MinContext);
i = 0;
num = p->MinContext->NumStats - numMasked;
do
{
int k = (int)(MASK(s->Symbol));
hiCnt += (s->Freq & k);
ps[i] = s++;
i -= k;
}
while (i != num);
see = Ppmd7_MakeEscFreq(p, numMasked, &freqSum);
freqSum += hiCnt;
count = rc->GetThreshold(rc, freqSum);
if (count < hiCnt)
{
Byte symbol;
CPpmd_State **pps = ps;
for (hiCnt = 0; (hiCnt += (*pps)->Freq) <= count; pps++);
s = *pps;
rc->Decode(rc, hiCnt - s->Freq, s->Freq);
Ppmd_See_Update(see);
p->FoundState = s;
symbol = s->Symbol;
Ppmd7_Update2(p);
return symbol;
}
if (count >= freqSum)
return -2;
rc->Decode(rc, hiCnt, freqSum - hiCnt);
see->Summ = (UInt16)(see->Summ + freqSum);
do { MASK(ps[--i]->Symbol) = 0; } while (i != 0);
}
}
+187
View File
@@ -0,0 +1,187 @@
/* Ppmd7Enc.c -- PPMdH Encoder
2017-04-03 : Igor Pavlov : Public domain
This code is based on PPMd var.H (2001): Dmitry Shkarin : Public domain */
#include "Precomp.h"
#include "Ppmd7.h"
#define kTopValue (1 << 24)
void Ppmd7z_RangeEnc_Init(CPpmd7z_RangeEnc *p)
{
p->Low = 0;
p->Range = 0xFFFFFFFF;
p->Cache = 0;
p->CacheSize = 1;
}
static void RangeEnc_ShiftLow(CPpmd7z_RangeEnc *p)
{
if ((UInt32)p->Low < (UInt32)0xFF000000 || (unsigned)(p->Low >> 32) != 0)
{
Byte temp = p->Cache;
do
{
IByteOut_Write(p->Stream, (Byte)(temp + (Byte)(p->Low >> 32)));
temp = 0xFF;
}
while (--p->CacheSize != 0);
p->Cache = (Byte)((UInt32)p->Low >> 24);
}
p->CacheSize++;
p->Low = (UInt32)p->Low << 8;
}
static void RangeEnc_Encode(CPpmd7z_RangeEnc *p, UInt32 start, UInt32 size, UInt32 total)
{
p->Low += start * (p->Range /= total);
p->Range *= size;
while (p->Range < kTopValue)
{
p->Range <<= 8;
RangeEnc_ShiftLow(p);
}
}
static void RangeEnc_EncodeBit_0(CPpmd7z_RangeEnc *p, UInt32 size0)
{
p->Range = (p->Range >> 14) * size0;
while (p->Range < kTopValue)
{
p->Range <<= 8;
RangeEnc_ShiftLow(p);
}
}
static void RangeEnc_EncodeBit_1(CPpmd7z_RangeEnc *p, UInt32 size0)
{
UInt32 newBound = (p->Range >> 14) * size0;
p->Low += newBound;
p->Range -= newBound;
while (p->Range < kTopValue)
{
p->Range <<= 8;
RangeEnc_ShiftLow(p);
}
}
void Ppmd7z_RangeEnc_FlushData(CPpmd7z_RangeEnc *p)
{
unsigned i;
for (i = 0; i < 5; i++)
RangeEnc_ShiftLow(p);
}
#define MASK(sym) ((signed char *)charMask)[sym]
void Ppmd7_EncodeSymbol(CPpmd7 *p, CPpmd7z_RangeEnc *rc, int symbol)
{
size_t charMask[256 / sizeof(size_t)];
if (p->MinContext->NumStats != 1)
{
CPpmd_State *s = Ppmd7_GetStats(p, p->MinContext);
UInt32 sum;
unsigned i;
if (s->Symbol == symbol)
{
RangeEnc_Encode(rc, 0, s->Freq, p->MinContext->SummFreq);
p->FoundState = s;
Ppmd7_Update1_0(p);
return;
}
p->PrevSuccess = 0;
sum = s->Freq;
i = p->MinContext->NumStats - 1;
do
{
if ((++s)->Symbol == symbol)
{
RangeEnc_Encode(rc, sum, s->Freq, p->MinContext->SummFreq);
p->FoundState = s;
Ppmd7_Update1(p);
return;
}
sum += s->Freq;
}
while (--i);
p->HiBitsFlag = p->HB2Flag[p->FoundState->Symbol];
PPMD_SetAllBitsIn256Bytes(charMask);
MASK(s->Symbol) = 0;
i = p->MinContext->NumStats - 1;
do { MASK((--s)->Symbol) = 0; } while (--i);
RangeEnc_Encode(rc, sum, p->MinContext->SummFreq - sum, p->MinContext->SummFreq);
}
else
{
UInt16 *prob = Ppmd7_GetBinSumm(p);
CPpmd_State *s = Ppmd7Context_OneState(p->MinContext);
if (s->Symbol == symbol)
{
RangeEnc_EncodeBit_0(rc, *prob);
*prob = (UInt16)PPMD_UPDATE_PROB_0(*prob);
p->FoundState = s;
Ppmd7_UpdateBin(p);
return;
}
else
{
RangeEnc_EncodeBit_1(rc, *prob);
*prob = (UInt16)PPMD_UPDATE_PROB_1(*prob);
p->InitEsc = PPMD7_kExpEscape[*prob >> 10];
PPMD_SetAllBitsIn256Bytes(charMask);
MASK(s->Symbol) = 0;
p->PrevSuccess = 0;
}
}
for (;;)
{
UInt32 escFreq;
CPpmd_See *see;
CPpmd_State *s;
UInt32 sum;
unsigned i, numMasked = p->MinContext->NumStats;
do
{
p->OrderFall++;
if (!p->MinContext->Suffix)
return; /* EndMarker (symbol = -1) */
p->MinContext = Ppmd7_GetContext(p, p->MinContext->Suffix);
}
while (p->MinContext->NumStats == numMasked);
see = Ppmd7_MakeEscFreq(p, numMasked, &escFreq);
s = Ppmd7_GetStats(p, p->MinContext);
sum = 0;
i = p->MinContext->NumStats;
do
{
int cur = s->Symbol;
if (cur == symbol)
{
UInt32 low = sum;
CPpmd_State *s1 = s;
do
{
sum += (s->Freq & (int)(MASK(s->Symbol)));
s++;
}
while (--i);
RangeEnc_Encode(rc, low, s1->Freq, sum + escFreq);
Ppmd_See_Update(see);
p->FoundState = s1;
Ppmd7_Update2(p);
return;
}
sum += (s->Freq & (int)(MASK(cur)));
MASK(cur) = 0;
s++;
}
while (--i);
RangeEnc_Encode(rc, sum, escFreq, sum + escFreq);
see->Summ = (UInt16)(see->Summ + sum + escFreq);
}
}
+10
View File
@@ -0,0 +1,10 @@
/* Precomp.h -- StdAfx
2013-11-12 : Igor Pavlov : Public domain */
#ifndef __7Z_PRECOMP_H
#define __7Z_PRECOMP_H
#include "Compiler.h"
/* #include "7zTypes.h" */
#endif
+30
View File
@@ -0,0 +1,30 @@
/* RotateDefs.h -- Rotate functions
2015-03-25 : Igor Pavlov : Public domain */
#ifndef __ROTATE_DEFS_H
#define __ROTATE_DEFS_H
#ifdef _MSC_VER
#include <stdlib.h>
/* don't use _rotl with MINGW. It can insert slow call to function. */
/* #if (_MSC_VER >= 1200) */
#pragma intrinsic(_rotl)
#pragma intrinsic(_rotr)
/* #endif */
#define rotlFixed(x, n) _rotl((x), (n))
#define rotrFixed(x, n) _rotr((x), (n))
#else
/* new compilers can translate these macros to fast commands. */
#define rotlFixed(x, n) (((x) << (n)) | ((x) >> (32 - (n))))
#define rotrFixed(x, n) (((x) >> (n)) | ((x) << (32 - (n))))
#endif
#endif
+248
View File
@@ -0,0 +1,248 @@
/* Crypto/Sha256.c -- SHA-256 Hash
2017-04-03 : Igor Pavlov : Public domain
This code is based on public domain code from Wei Dai's Crypto++ library. */
#include "Precomp.h"
#include <string.h>
#include "CpuArch.h"
#include "RotateDefs.h"
#include "Sha256.h"
/* define it for speed optimization */
#ifndef _SFX
#define _SHA256_UNROLL
#define _SHA256_UNROLL2
#endif
/* #define _SHA256_UNROLL2 */
void Sha256_Init(CSha256 *p)
{
p->state[0] = 0x6a09e667;
p->state[1] = 0xbb67ae85;
p->state[2] = 0x3c6ef372;
p->state[3] = 0xa54ff53a;
p->state[4] = 0x510e527f;
p->state[5] = 0x9b05688c;
p->state[6] = 0x1f83d9ab;
p->state[7] = 0x5be0cd19;
p->count = 0;
}
#define S0(x) (rotrFixed(x, 2) ^ rotrFixed(x,13) ^ rotrFixed(x, 22))
#define S1(x) (rotrFixed(x, 6) ^ rotrFixed(x,11) ^ rotrFixed(x, 25))
#define s0(x) (rotrFixed(x, 7) ^ rotrFixed(x,18) ^ (x >> 3))
#define s1(x) (rotrFixed(x,17) ^ rotrFixed(x,19) ^ (x >> 10))
#define blk0(i) (W[i])
#define blk2(i) (W[i] += s1(W[((i)-2)&15]) + W[((i)-7)&15] + s0(W[((i)-15)&15]))
#define Ch(x,y,z) (z^(x&(y^z)))
#define Maj(x,y,z) ((x&y)|(z&(x|y)))
#ifdef _SHA256_UNROLL2
#define R(a,b,c,d,e,f,g,h, i) \
h += S1(e) + Ch(e,f,g) + K[(i)+(size_t)(j)] + (j ? blk2(i) : blk0(i)); \
d += h; \
h += S0(a) + Maj(a, b, c)
#define RX_8(i) \
R(a,b,c,d,e,f,g,h, i); \
R(h,a,b,c,d,e,f,g, i+1); \
R(g,h,a,b,c,d,e,f, i+2); \
R(f,g,h,a,b,c,d,e, i+3); \
R(e,f,g,h,a,b,c,d, i+4); \
R(d,e,f,g,h,a,b,c, i+5); \
R(c,d,e,f,g,h,a,b, i+6); \
R(b,c,d,e,f,g,h,a, i+7)
#define RX_16 RX_8(0); RX_8(8);
#else
#define a(i) T[(0-(i))&7]
#define b(i) T[(1-(i))&7]
#define c(i) T[(2-(i))&7]
#define d(i) T[(3-(i))&7]
#define e(i) T[(4-(i))&7]
#define f(i) T[(5-(i))&7]
#define g(i) T[(6-(i))&7]
#define h(i) T[(7-(i))&7]
#define R(i) \
h(i) += S1(e(i)) + Ch(e(i),f(i),g(i)) + K[(i)+(size_t)(j)] + (j ? blk2(i) : blk0(i)); \
d(i) += h(i); \
h(i) += S0(a(i)) + Maj(a(i), b(i), c(i)) \
#ifdef _SHA256_UNROLL
#define RX_8(i) R(i+0); R(i+1); R(i+2); R(i+3); R(i+4); R(i+5); R(i+6); R(i+7);
#define RX_16 RX_8(0); RX_8(8);
#else
#define RX_16 unsigned i; for (i = 0; i < 16; i++) { R(i); }
#endif
#endif
static const UInt32 K[64] = {
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5,
0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3,
0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc,
0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7,
0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13,
0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3,
0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5,
0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208,
0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2
};
static void Sha256_WriteByteBlock(CSha256 *p)
{
UInt32 W[16];
unsigned j;
UInt32 *state;
#ifdef _SHA256_UNROLL2
UInt32 a,b,c,d,e,f,g,h;
#else
UInt32 T[8];
#endif
for (j = 0; j < 16; j += 4)
{
const Byte *ccc = p->buffer + j * 4;
W[j ] = GetBe32(ccc);
W[j + 1] = GetBe32(ccc + 4);
W[j + 2] = GetBe32(ccc + 8);
W[j + 3] = GetBe32(ccc + 12);
}
state = p->state;
#ifdef _SHA256_UNROLL2
a = state[0];
b = state[1];
c = state[2];
d = state[3];
e = state[4];
f = state[5];
g = state[6];
h = state[7];
#else
for (j = 0; j < 8; j++)
T[j] = state[j];
#endif
for (j = 0; j < 64; j += 16)
{
RX_16
}
#ifdef _SHA256_UNROLL2
state[0] += a;
state[1] += b;
state[2] += c;
state[3] += d;
state[4] += e;
state[5] += f;
state[6] += g;
state[7] += h;
#else
for (j = 0; j < 8; j++)
state[j] += T[j];
#endif
/* Wipe variables */
/* memset(W, 0, sizeof(W)); */
/* memset(T, 0, sizeof(T)); */
}
#undef S0
#undef S1
#undef s0
#undef s1
void Sha256_Update(CSha256 *p, const Byte *data, size_t size)
{
if (size == 0)
return;
{
unsigned pos = (unsigned)p->count & 0x3F;
unsigned num;
p->count += size;
num = 64 - pos;
if (num > size)
{
memcpy(p->buffer + pos, data, size);
return;
}
size -= num;
memcpy(p->buffer + pos, data, num);
data += num;
}
for (;;)
{
Sha256_WriteByteBlock(p);
if (size < 64)
break;
size -= 64;
memcpy(p->buffer, data, 64);
data += 64;
}
if (size != 0)
memcpy(p->buffer, data, size);
}
void Sha256_Final(CSha256 *p, Byte *digest)
{
unsigned pos = (unsigned)p->count & 0x3F;
unsigned i;
p->buffer[pos++] = 0x80;
while (pos != (64 - 8))
{
pos &= 0x3F;
if (pos == 0)
Sha256_WriteByteBlock(p);
p->buffer[pos++] = 0;
}
{
UInt64 numBits = (p->count << 3);
SetBe32(p->buffer + 64 - 8, (UInt32)(numBits >> 32));
SetBe32(p->buffer + 64 - 4, (UInt32)(numBits));
}
Sha256_WriteByteBlock(p);
for (i = 0; i < 8; i += 2)
{
UInt32 v0 = p->state[i];
UInt32 v1 = p->state[i + 1];
SetBe32(digest , v0);
SetBe32(digest + 4, v1);
digest += 8;
}
Sha256_Init(p);
}
+26
View File
@@ -0,0 +1,26 @@
/* Sha256.h -- SHA-256 Hash
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __CRYPTO_SHA256_H
#define __CRYPTO_SHA256_H
#include "7zTypes.h"
EXTERN_C_BEGIN
#define SHA256_DIGEST_SIZE 32
typedef struct
{
UInt32 state[8];
UInt64 count;
Byte buffer[64];
} CSha256;
void Sha256_Init(CSha256 *p);
void Sha256_Update(CSha256 *p, const Byte *data, size_t size);
void Sha256_Final(CSha256 *p, Byte *digest);
EXTERN_C_END
#endif
+141
View File
@@ -0,0 +1,141 @@
/* Sort.c -- Sort functions
2014-04-05 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "Sort.h"
#define HeapSortDown(p, k, size, temp) \
{ for (;;) { \
size_t s = (k << 1); \
if (s > size) break; \
if (s < size && p[s + 1] > p[s]) s++; \
if (temp >= p[s]) break; \
p[k] = p[s]; k = s; \
} p[k] = temp; }
void HeapSort(UInt32 *p, size_t size)
{
if (size <= 1)
return;
p--;
{
size_t i = size / 2;
do
{
UInt32 temp = p[i];
size_t k = i;
HeapSortDown(p, k, size, temp)
}
while (--i != 0);
}
/*
do
{
size_t k = 1;
UInt32 temp = p[size];
p[size--] = p[1];
HeapSortDown(p, k, size, temp)
}
while (size > 1);
*/
while (size > 3)
{
UInt32 temp = p[size];
size_t k = (p[3] > p[2]) ? 3 : 2;
p[size--] = p[1];
p[1] = p[k];
HeapSortDown(p, k, size, temp)
}
{
UInt32 temp = p[size];
p[size] = p[1];
if (size > 2 && p[2] < temp)
{
p[1] = p[2];
p[2] = temp;
}
else
p[1] = temp;
}
}
void HeapSort64(UInt64 *p, size_t size)
{
if (size <= 1)
return;
p--;
{
size_t i = size / 2;
do
{
UInt64 temp = p[i];
size_t k = i;
HeapSortDown(p, k, size, temp)
}
while (--i != 0);
}
/*
do
{
size_t k = 1;
UInt64 temp = p[size];
p[size--] = p[1];
HeapSortDown(p, k, size, temp)
}
while (size > 1);
*/
while (size > 3)
{
UInt64 temp = p[size];
size_t k = (p[3] > p[2]) ? 3 : 2;
p[size--] = p[1];
p[1] = p[k];
HeapSortDown(p, k, size, temp)
}
{
UInt64 temp = p[size];
p[size] = p[1];
if (size > 2 && p[2] < temp)
{
p[1] = p[2];
p[2] = temp;
}
else
p[1] = temp;
}
}
/*
#define HeapSortRefDown(p, vals, n, size, temp) \
{ size_t k = n; UInt32 val = vals[temp]; for (;;) { \
size_t s = (k << 1); \
if (s > size) break; \
if (s < size && vals[p[s + 1]] > vals[p[s]]) s++; \
if (val >= vals[p[s]]) break; \
p[k] = p[s]; k = s; \
} p[k] = temp; }
void HeapSortRef(UInt32 *p, UInt32 *vals, size_t size)
{
if (size <= 1)
return;
p--;
{
size_t i = size / 2;
do
{
UInt32 temp = p[i];
HeapSortRefDown(p, vals, i, size, temp);
}
while (--i != 0);
}
do
{
UInt32 temp = p[size];
p[size--] = p[1];
HeapSortRefDown(p, vals, 1, size, temp);
}
while (size > 1);
}
*/
+18
View File
@@ -0,0 +1,18 @@
/* Sort.h -- Sort functions
2014-04-05 : Igor Pavlov : Public domain */
#ifndef __7Z_SORT_H
#define __7Z_SORT_H
#include "7zTypes.h"
EXTERN_C_BEGIN
void HeapSort(UInt32 *p, size_t size);
void HeapSort64(UInt64 *p, size_t size);
/* void HeapSortRef(UInt32 *p, UInt32 *vals, size_t size); */
EXTERN_C_END
#endif
+95
View File
@@ -0,0 +1,95 @@
/* Threads.c -- multithreading library
2017-06-26 : Igor Pavlov : Public domain */
#include "Precomp.h"
#ifndef UNDER_CE
#include <process.h>
#endif
#include "Threads.h"
static WRes GetError()
{
DWORD res = GetLastError();
return res ? (WRes)res : 1;
}
static WRes HandleToWRes(HANDLE h) { return (h != NULL) ? 0 : GetError(); }
static WRes BOOLToWRes(BOOL v) { return v ? 0 : GetError(); }
WRes HandlePtr_Close(HANDLE *p)
{
if (*p != NULL)
{
if (!CloseHandle(*p))
return GetError();
*p = NULL;
}
return 0;
}
WRes Handle_WaitObject(HANDLE h) { return (WRes)WaitForSingleObject(h, INFINITE); }
WRes Thread_Create(CThread *p, THREAD_FUNC_TYPE func, LPVOID param)
{
/* Windows Me/98/95: threadId parameter may not be NULL in _beginthreadex/CreateThread functions */
#ifdef UNDER_CE
DWORD threadId;
*p = CreateThread(0, 0, func, param, 0, &threadId);
#else
unsigned threadId;
*p = (HANDLE)_beginthreadex(NULL, 0, func, param, 0, &threadId);
#endif
/* maybe we must use errno here, but probably GetLastError() is also OK. */
return HandleToWRes(*p);
}
static WRes Event_Create(CEvent *p, BOOL manualReset, int signaled)
{
*p = CreateEvent(NULL, manualReset, (signaled ? TRUE : FALSE), NULL);
return HandleToWRes(*p);
}
WRes Event_Set(CEvent *p) { return BOOLToWRes(SetEvent(*p)); }
WRes Event_Reset(CEvent *p) { return BOOLToWRes(ResetEvent(*p)); }
WRes ManualResetEvent_Create(CManualResetEvent *p, int signaled) { return Event_Create(p, TRUE, signaled); }
WRes AutoResetEvent_Create(CAutoResetEvent *p, int signaled) { return Event_Create(p, FALSE, signaled); }
WRes ManualResetEvent_CreateNotSignaled(CManualResetEvent *p) { return ManualResetEvent_Create(p, 0); }
WRes AutoResetEvent_CreateNotSignaled(CAutoResetEvent *p) { return AutoResetEvent_Create(p, 0); }
WRes Semaphore_Create(CSemaphore *p, UInt32 initCount, UInt32 maxCount)
{
*p = CreateSemaphore(NULL, (LONG)initCount, (LONG)maxCount, NULL);
return HandleToWRes(*p);
}
static WRes Semaphore_Release(CSemaphore *p, LONG releaseCount, LONG *previousCount)
{ return BOOLToWRes(ReleaseSemaphore(*p, releaseCount, previousCount)); }
WRes Semaphore_ReleaseN(CSemaphore *p, UInt32 num)
{ return Semaphore_Release(p, (LONG)num, NULL); }
WRes Semaphore_Release1(CSemaphore *p) { return Semaphore_ReleaseN(p, 1); }
WRes CriticalSection_Init(CCriticalSection *p)
{
/* InitializeCriticalSection can raise only STATUS_NO_MEMORY exception */
#ifdef _MSC_VER
__try
#endif
{
InitializeCriticalSection(p);
/* InitializeCriticalSectionAndSpinCount(p, 0); */
}
#ifdef _MSC_VER
__except (EXCEPTION_EXECUTE_HANDLER) { return 1; }
#endif
return 0;
}
+68
View File
@@ -0,0 +1,68 @@
/* Threads.h -- multithreading library
2017-06-18 : Igor Pavlov : Public domain */
#ifndef __7Z_THREADS_H
#define __7Z_THREADS_H
#ifdef _WIN32
#include <windows.h>
#endif
#include "7zTypes.h"
EXTERN_C_BEGIN
WRes HandlePtr_Close(HANDLE *h);
WRes Handle_WaitObject(HANDLE h);
typedef HANDLE CThread;
#define Thread_Construct(p) *(p) = NULL
#define Thread_WasCreated(p) (*(p) != NULL)
#define Thread_Close(p) HandlePtr_Close(p)
#define Thread_Wait(p) Handle_WaitObject(*(p))
typedef
#ifdef UNDER_CE
DWORD
#else
unsigned
#endif
THREAD_FUNC_RET_TYPE;
#define THREAD_FUNC_CALL_TYPE MY_STD_CALL
#define THREAD_FUNC_DECL THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE
typedef THREAD_FUNC_RET_TYPE (THREAD_FUNC_CALL_TYPE * THREAD_FUNC_TYPE)(void *);
WRes Thread_Create(CThread *p, THREAD_FUNC_TYPE func, LPVOID param);
typedef HANDLE CEvent;
typedef CEvent CAutoResetEvent;
typedef CEvent CManualResetEvent;
#define Event_Construct(p) *(p) = NULL
#define Event_IsCreated(p) (*(p) != NULL)
#define Event_Close(p) HandlePtr_Close(p)
#define Event_Wait(p) Handle_WaitObject(*(p))
WRes Event_Set(CEvent *p);
WRes Event_Reset(CEvent *p);
WRes ManualResetEvent_Create(CManualResetEvent *p, int signaled);
WRes ManualResetEvent_CreateNotSignaled(CManualResetEvent *p);
WRes AutoResetEvent_Create(CAutoResetEvent *p, int signaled);
WRes AutoResetEvent_CreateNotSignaled(CAutoResetEvent *p);
typedef HANDLE CSemaphore;
#define Semaphore_Construct(p) *(p) = NULL
#define Semaphore_IsCreated(p) (*(p) != NULL)
#define Semaphore_Close(p) HandlePtr_Close(p)
#define Semaphore_Wait(p) Handle_WaitObject(*(p))
WRes Semaphore_Create(CSemaphore *p, UInt32 initCount, UInt32 maxCount);
WRes Semaphore_ReleaseN(CSemaphore *p, UInt32 num);
WRes Semaphore_Release1(CSemaphore *p);
typedef CRITICAL_SECTION CCriticalSection;
WRes CriticalSection_Init(CCriticalSection *p);
#define CriticalSection_Delete(p) DeleteCriticalSection(p)
#define CriticalSection_Enter(p) EnterCriticalSection(p)
#define CriticalSection_Leave(p) LeaveCriticalSection(p)
EXTERN_C_END
#endif
+90
View File
@@ -0,0 +1,90 @@
/* Xz.c - Xz
2017-05-12 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "7zCrc.h"
#include "CpuArch.h"
#include "Xz.h"
#include "XzCrc64.h"
const Byte XZ_SIG[XZ_SIG_SIZE] = { 0xFD, '7', 'z', 'X', 'Z', 0 };
/* const Byte XZ_FOOTER_SIG[XZ_FOOTER_SIG_SIZE] = { 'Y', 'Z' }; */
unsigned Xz_WriteVarInt(Byte *buf, UInt64 v)
{
unsigned i = 0;
do
{
buf[i++] = (Byte)((v & 0x7F) | 0x80);
v >>= 7;
}
while (v != 0);
buf[(size_t)i - 1] &= 0x7F;
return i;
}
void Xz_Construct(CXzStream *p)
{
p->numBlocks = 0;
p->blocks = NULL;
p->flags = 0;
}
void Xz_Free(CXzStream *p, ISzAllocPtr alloc)
{
ISzAlloc_Free(alloc, p->blocks);
p->numBlocks = 0;
p->blocks = NULL;
}
unsigned XzFlags_GetCheckSize(CXzStreamFlags f)
{
unsigned t = XzFlags_GetCheckType(f);
return (t == 0) ? 0 : (4 << ((t - 1) / 3));
}
void XzCheck_Init(CXzCheck *p, unsigned mode)
{
p->mode = mode;
switch (mode)
{
case XZ_CHECK_CRC32: p->crc = CRC_INIT_VAL; break;
case XZ_CHECK_CRC64: p->crc64 = CRC64_INIT_VAL; break;
case XZ_CHECK_SHA256: Sha256_Init(&p->sha); break;
}
}
void XzCheck_Update(CXzCheck *p, const void *data, size_t size)
{
switch (p->mode)
{
case XZ_CHECK_CRC32: p->crc = CrcUpdate(p->crc, data, size); break;
case XZ_CHECK_CRC64: p->crc64 = Crc64Update(p->crc64, data, size); break;
case XZ_CHECK_SHA256: Sha256_Update(&p->sha, (const Byte *)data, size); break;
}
}
int XzCheck_Final(CXzCheck *p, Byte *digest)
{
switch (p->mode)
{
case XZ_CHECK_CRC32:
SetUi32(digest, CRC_GET_DIGEST(p->crc));
break;
case XZ_CHECK_CRC64:
{
int i;
UInt64 v = CRC64_GET_DIGEST(p->crc64);
for (i = 0; i < 8; i++, v >>= 8)
digest[i] = (Byte)(v & 0xFF);
break;
}
case XZ_CHECK_SHA256:
Sha256_Final(&p->sha, digest);
break;
default:
return 0;
}
return 1;
}
+460
View File
@@ -0,0 +1,460 @@
/* Xz.h - Xz interface
2018-07-04 : Igor Pavlov : Public domain */
#ifndef __XZ_H
#define __XZ_H
#include "Sha256.h"
EXTERN_C_BEGIN
#define XZ_ID_Subblock 1
#define XZ_ID_Delta 3
#define XZ_ID_X86 4
#define XZ_ID_PPC 5
#define XZ_ID_IA64 6
#define XZ_ID_ARM 7
#define XZ_ID_ARMT 8
#define XZ_ID_SPARC 9
#define XZ_ID_LZMA2 0x21
unsigned Xz_ReadVarInt(const Byte *p, size_t maxSize, UInt64 *value);
unsigned Xz_WriteVarInt(Byte *buf, UInt64 v);
/* ---------- xz block ---------- */
#define XZ_BLOCK_HEADER_SIZE_MAX 1024
#define XZ_NUM_FILTERS_MAX 4
#define XZ_BF_NUM_FILTERS_MASK 3
#define XZ_BF_PACK_SIZE (1 << 6)
#define XZ_BF_UNPACK_SIZE (1 << 7)
#define XZ_FILTER_PROPS_SIZE_MAX 20
typedef struct
{
UInt64 id;
UInt32 propsSize;
Byte props[XZ_FILTER_PROPS_SIZE_MAX];
} CXzFilter;
typedef struct
{
UInt64 packSize;
UInt64 unpackSize;
Byte flags;
CXzFilter filters[XZ_NUM_FILTERS_MAX];
} CXzBlock;
#define XzBlock_GetNumFilters(p) (((p)->flags & XZ_BF_NUM_FILTERS_MASK) + 1)
#define XzBlock_HasPackSize(p) (((p)->flags & XZ_BF_PACK_SIZE) != 0)
#define XzBlock_HasUnpackSize(p) (((p)->flags & XZ_BF_UNPACK_SIZE) != 0)
#define XzBlock_HasUnsupportedFlags(p) (((p)->flags & ~(XZ_BF_NUM_FILTERS_MASK | XZ_BF_PACK_SIZE | XZ_BF_UNPACK_SIZE)) != 0)
SRes XzBlock_Parse(CXzBlock *p, const Byte *header);
SRes XzBlock_ReadHeader(CXzBlock *p, ISeqInStream *inStream, BoolInt *isIndex, UInt32 *headerSizeRes);
/* ---------- xz stream ---------- */
#define XZ_SIG_SIZE 6
#define XZ_FOOTER_SIG_SIZE 2
extern const Byte XZ_SIG[XZ_SIG_SIZE];
/*
extern const Byte XZ_FOOTER_SIG[XZ_FOOTER_SIG_SIZE];
*/
#define XZ_FOOTER_SIG_0 'Y'
#define XZ_FOOTER_SIG_1 'Z'
#define XZ_STREAM_FLAGS_SIZE 2
#define XZ_STREAM_CRC_SIZE 4
#define XZ_STREAM_HEADER_SIZE (XZ_SIG_SIZE + XZ_STREAM_FLAGS_SIZE + XZ_STREAM_CRC_SIZE)
#define XZ_STREAM_FOOTER_SIZE (XZ_FOOTER_SIG_SIZE + XZ_STREAM_FLAGS_SIZE + XZ_STREAM_CRC_SIZE + 4)
#define XZ_CHECK_MASK 0xF
#define XZ_CHECK_NO 0
#define XZ_CHECK_CRC32 1
#define XZ_CHECK_CRC64 4
#define XZ_CHECK_SHA256 10
typedef struct
{
unsigned mode;
UInt32 crc;
UInt64 crc64;
CSha256 sha;
} CXzCheck;
void XzCheck_Init(CXzCheck *p, unsigned mode);
void XzCheck_Update(CXzCheck *p, const void *data, size_t size);
int XzCheck_Final(CXzCheck *p, Byte *digest);
typedef UInt16 CXzStreamFlags;
#define XzFlags_IsSupported(f) ((f) <= XZ_CHECK_MASK)
#define XzFlags_GetCheckType(f) ((f) & XZ_CHECK_MASK)
#define XzFlags_HasDataCrc32(f) (Xz_GetCheckType(f) == XZ_CHECK_CRC32)
unsigned XzFlags_GetCheckSize(CXzStreamFlags f);
SRes Xz_ParseHeader(CXzStreamFlags *p, const Byte *buf);
SRes Xz_ReadHeader(CXzStreamFlags *p, ISeqInStream *inStream);
typedef struct
{
UInt64 unpackSize;
UInt64 totalSize;
} CXzBlockSizes;
typedef struct
{
CXzStreamFlags flags;
size_t numBlocks;
CXzBlockSizes *blocks;
UInt64 startOffset;
} CXzStream;
void Xz_Construct(CXzStream *p);
void Xz_Free(CXzStream *p, ISzAllocPtr alloc);
#define XZ_SIZE_OVERFLOW ((UInt64)(Int64)-1)
UInt64 Xz_GetUnpackSize(const CXzStream *p);
UInt64 Xz_GetPackSize(const CXzStream *p);
typedef struct
{
size_t num;
size_t numAllocated;
CXzStream *streams;
} CXzs;
void Xzs_Construct(CXzs *p);
void Xzs_Free(CXzs *p, ISzAllocPtr alloc);
SRes Xzs_ReadBackward(CXzs *p, ILookInStream *inStream, Int64 *startOffset, ICompressProgress *progress, ISzAllocPtr alloc);
UInt64 Xzs_GetNumBlocks(const CXzs *p);
UInt64 Xzs_GetUnpackSize(const CXzs *p);
// ECoderStatus values are identical to ELzmaStatus values of LZMA2 decoder
typedef enum
{
CODER_STATUS_NOT_SPECIFIED, /* use main error code instead */
CODER_STATUS_FINISHED_WITH_MARK, /* stream was finished with end mark. */
CODER_STATUS_NOT_FINISHED, /* stream was not finished */
CODER_STATUS_NEEDS_MORE_INPUT /* you must provide more input bytes */
} ECoderStatus;
// ECoderFinishMode values are identical to ELzmaFinishMode
typedef enum
{
CODER_FINISH_ANY, /* finish at any point */
CODER_FINISH_END /* block must be finished at the end */
} ECoderFinishMode;
typedef struct _IStateCoder
{
void *p;
void (*Free)(void *p, ISzAllocPtr alloc);
SRes (*SetProps)(void *p, const Byte *props, size_t propSize, ISzAllocPtr alloc);
void (*Init)(void *p);
SRes (*Code2)(void *p, Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
int srcWasFinished, ECoderFinishMode finishMode,
// int *wasFinished,
ECoderStatus *status);
SizeT (*Filter)(void *p, Byte *data, SizeT size);
} IStateCoder;
#define MIXCODER_NUM_FILTERS_MAX 4
typedef struct
{
ISzAllocPtr alloc;
Byte *buf;
unsigned numCoders;
Byte *outBuf;
size_t outBufSize;
size_t outWritten; // is equal to lzmaDecoder.dicPos (in outBuf mode)
BoolInt wasFinished;
SRes res;
ECoderStatus status;
// BoolInt SingleBufMode;
int finished[MIXCODER_NUM_FILTERS_MAX - 1];
size_t pos[MIXCODER_NUM_FILTERS_MAX - 1];
size_t size[MIXCODER_NUM_FILTERS_MAX - 1];
UInt64 ids[MIXCODER_NUM_FILTERS_MAX];
SRes results[MIXCODER_NUM_FILTERS_MAX];
IStateCoder coders[MIXCODER_NUM_FILTERS_MAX];
} CMixCoder;
typedef enum
{
XZ_STATE_STREAM_HEADER,
XZ_STATE_STREAM_INDEX,
XZ_STATE_STREAM_INDEX_CRC,
XZ_STATE_STREAM_FOOTER,
XZ_STATE_STREAM_PADDING,
XZ_STATE_BLOCK_HEADER,
XZ_STATE_BLOCK,
XZ_STATE_BLOCK_FOOTER
} EXzState;
typedef struct
{
EXzState state;
UInt32 pos;
unsigned alignPos;
unsigned indexPreSize;
CXzStreamFlags streamFlags;
UInt32 blockHeaderSize;
UInt64 packSize;
UInt64 unpackSize;
UInt64 numBlocks; // number of finished blocks in current stream
UInt64 indexSize;
UInt64 indexPos;
UInt64 padSize;
UInt64 numStartedStreams;
UInt64 numFinishedStreams;
UInt64 numTotalBlocks;
UInt32 crc;
CMixCoder decoder;
CXzBlock block;
CXzCheck check;
CSha256 sha;
BoolInt parseMode;
BoolInt headerParsedOk;
BoolInt decodeToStreamSignature;
unsigned decodeOnlyOneBlock;
Byte *outBuf;
size_t outBufSize;
size_t outDataWritten; // the size of data in (outBuf) that were fully unpacked
Byte shaDigest[SHA256_DIGEST_SIZE];
Byte buf[XZ_BLOCK_HEADER_SIZE_MAX];
} CXzUnpacker;
/* alloc : aligned for cache line allocation is better */
void XzUnpacker_Construct(CXzUnpacker *p, ISzAllocPtr alloc);
void XzUnpacker_Init(CXzUnpacker *p);
void XzUnpacker_SetOutBuf(CXzUnpacker *p, Byte *outBuf, size_t outBufSize);
void XzUnpacker_Free(CXzUnpacker *p);
/*
XzUnpacker
The sequence for decoding functions:
{
XzUnpacker_Construct()
[Decoding_Calls]
XzUnpacker_Free()
}
[Decoding_Calls]
There are 3 types of interfaces for [Decoding_Calls] calls:
Interface-1 : Partial output buffers:
{
XzUnpacker_Init()
for()
XzUnpacker_Code();
}
Interface-2 : Direct output buffer:
Use it, if you know exact size of decoded data, and you need
whole xz unpacked data in one output buffer.
xz unpacker doesn't allocate additional buffer for lzma2 dictionary in that mode.
{
XzUnpacker_Init()
XzUnpacker_SetOutBufMode(); // to set output buffer and size
for()
XzUnpacker_Code(); // (dest = NULL) in XzUnpacker_Code()
}
Interface-3 : Direct output buffer : One call full decoding
It unpacks whole input buffer to output buffer in one call.
It uses Interface-2 internally.
{
XzUnpacker_CodeFull()
}
*/
/*
finishMode:
It has meaning only if the decoding reaches output limit (*destLen).
CODER_FINISH_ANY - use smallest number of input bytes
CODER_FINISH_END - read EndOfStream marker after decoding
Returns:
SZ_OK
status:
CODER_STATUS_NOT_FINISHED,
CODER_STATUS_NEEDS_MORE_INPUT - maybe there are more xz streams,
call XzUnpacker_IsStreamWasFinished to check that current stream was finished
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_DATA - Data error
SZ_ERROR_UNSUPPORTED - Unsupported method or method properties
SZ_ERROR_CRC - CRC error
// SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src).
SZ_ERROR_NO_ARCHIVE - the error with xz Stream Header with one of the following reasons:
- xz Stream Signature failure
- CRC32 of xz Stream Header is failed
- The size of Stream padding is not multiple of four bytes.
It's possible to get that error, if xz stream was finished and the stream
contains some another data. In that case you can call XzUnpacker_GetExtraSize()
function to get real size of xz stream.
*/
SRes XzUnpacker_Code(CXzUnpacker *p, Byte *dest, SizeT *destLen,
const Byte *src, SizeT *srcLen, int srcFinished,
ECoderFinishMode finishMode, ECoderStatus *status);
SRes XzUnpacker_CodeFull(CXzUnpacker *p, Byte *dest, SizeT *destLen,
const Byte *src, SizeT *srcLen,
ECoderFinishMode finishMode, ECoderStatus *status);
BoolInt XzUnpacker_IsStreamWasFinished(const CXzUnpacker *p);
/*
XzUnpacker_GetExtraSize() returns then number of uncofirmed bytes,
if it's in (XZ_STATE_STREAM_HEADER) state or in (XZ_STATE_STREAM_PADDING) state.
These bytes can be some bytes after xz archive, or
it can be start of new xz stream.
Call XzUnpacker_GetExtraSize() after XzUnpacker_Code() function to detect real size of
xz stream in two cases, if XzUnpacker_Code() returns:
res == SZ_OK && status == CODER_STATUS_NEEDS_MORE_INPUT
res == SZ_ERROR_NO_ARCHIVE
*/
UInt64 XzUnpacker_GetExtraSize(const CXzUnpacker *p);
/*
for random block decoding:
XzUnpacker_Init();
set CXzUnpacker::streamFlags
XzUnpacker_PrepareToRandomBlockDecoding()
loop
{
XzUnpacker_Code()
XzUnpacker_IsBlockFinished()
}
*/
void XzUnpacker_PrepareToRandomBlockDecoding(CXzUnpacker *p);
BoolInt XzUnpacker_IsBlockFinished(const CXzUnpacker *p);
#define XzUnpacker_GetPackSizeForIndex(p) ((p)->packSize + (p)->blockHeaderSize + XzFlags_GetCheckSize((p)->streamFlags))
/* ---------- Multi Threading Decoding ---------- */
typedef struct
{
size_t inBufSize_ST;
size_t outStep_ST;
BoolInt ignoreErrors;
#ifndef _7ZIP_ST
unsigned numThreads;
size_t inBufSize_MT;
size_t memUseMax;
#endif
} CXzDecMtProps;
void XzDecMtProps_Init(CXzDecMtProps *p);
typedef void * CXzDecMtHandle;
/*
alloc : XzDecMt uses CAlignOffsetAlloc for addresses allocated by (alloc).
allocMid : for big allocations, aligned allocation is better
*/
CXzDecMtHandle XzDecMt_Create(ISzAllocPtr alloc, ISzAllocPtr allocMid);
void XzDecMt_Destroy(CXzDecMtHandle p);
typedef struct
{
Byte UnpackSize_Defined;
Byte NumStreams_Defined;
Byte NumBlocks_Defined;
Byte DataAfterEnd;
Byte DecodingTruncated; // Decoding was Truncated, we need only partial output data
UInt64 InSize; // pack size processed
UInt64 OutSize;
UInt64 NumStreams;
UInt64 NumBlocks;
SRes DecodeRes;
SRes ReadRes;
SRes ProgressRes;
SRes CombinedRes;
SRes CombinedRes_Type;
} CXzStatInfo;
void XzStatInfo_Clear(CXzStatInfo *p);
/*
XzDecMt_Decode()
SRes:
SZ_OK - OK
SZ_ERROR_MEM - Memory allocation error
SZ_ERROR_NO_ARCHIVE - is not xz archive
SZ_ERROR_ARCHIVE - Headers error
SZ_ERROR_DATA - Data Error
SZ_ERROR_CRC - CRC Error
SZ_ERROR_INPUT_EOF - it needs more input data
SZ_ERROR_WRITE - ISeqOutStream error
(SZ_ERROR_READ) - ISeqInStream errors
(SZ_ERROR_PROGRESS) - ICompressProgress errors
// SZ_ERROR_THREAD - error in multi-threading functions
MY_SRes_HRESULT_FROM_WRes(WRes_error) - error in multi-threading function
*/
SRes XzDecMt_Decode(CXzDecMtHandle p,
const CXzDecMtProps *props,
const UInt64 *outDataSize, // NULL means undefined
int finishMode, // 0 - partial unpacking is allowed, 1 - xz stream(s) must be finished
ISeqOutStream *outStream,
// Byte *outBuf, size_t *outBufSize,
ISeqInStream *inStream,
// const Byte *inData, size_t inDataSize,
CXzStatInfo *stat,
int *isMT, // 0 means that ST (Single-Thread) version was used
ICompressProgress *progress);
EXTERN_C_END
#endif
+86
View File
@@ -0,0 +1,86 @@
/* XzCrc64.c -- CRC64 calculation
2017-05-23 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "XzCrc64.h"
#include "CpuArch.h"
#define kCrc64Poly UINT64_CONST(0xC96C5795D7870F42)
#ifdef MY_CPU_LE
#define CRC64_NUM_TABLES 4
#else
#define CRC64_NUM_TABLES 5
#define CRC_UINT64_SWAP(v) \
((v >> 56) \
| ((v >> 40) & ((UInt64)0xFF << 8)) \
| ((v >> 24) & ((UInt64)0xFF << 16)) \
| ((v >> 8) & ((UInt64)0xFF << 24)) \
| ((v << 8) & ((UInt64)0xFF << 32)) \
| ((v << 24) & ((UInt64)0xFF << 40)) \
| ((v << 40) & ((UInt64)0xFF << 48)) \
| ((v << 56)))
UInt64 MY_FAST_CALL XzCrc64UpdateT1_BeT4(UInt64 v, const void *data, size_t size, const UInt64 *table);
#endif
#ifndef MY_CPU_BE
UInt64 MY_FAST_CALL XzCrc64UpdateT4(UInt64 v, const void *data, size_t size, const UInt64 *table);
#endif
typedef UInt64 (MY_FAST_CALL *CRC64_FUNC)(UInt64 v, const void *data, size_t size, const UInt64 *table);
static CRC64_FUNC g_Crc64Update;
UInt64 g_Crc64Table[256 * CRC64_NUM_TABLES];
UInt64 MY_FAST_CALL Crc64Update(UInt64 v, const void *data, size_t size)
{
return g_Crc64Update(v, data, size, g_Crc64Table);
}
UInt64 MY_FAST_CALL Crc64Calc(const void *data, size_t size)
{
return g_Crc64Update(CRC64_INIT_VAL, data, size, g_Crc64Table) ^ CRC64_INIT_VAL;
}
void MY_FAST_CALL Crc64GenerateTable()
{
UInt32 i;
for (i = 0; i < 256; i++)
{
UInt64 r = i;
unsigned j;
for (j = 0; j < 8; j++)
r = (r >> 1) ^ (kCrc64Poly & ((UInt64)0 - (r & 1)));
g_Crc64Table[i] = r;
}
for (i = 256; i < 256 * CRC64_NUM_TABLES; i++)
{
UInt64 r = g_Crc64Table[(size_t)i - 256];
g_Crc64Table[i] = g_Crc64Table[r & 0xFF] ^ (r >> 8);
}
#ifdef MY_CPU_LE
g_Crc64Update = XzCrc64UpdateT4;
#else
{
#ifndef MY_CPU_BE
UInt32 k = 1;
if (*(const Byte *)&k == 1)
g_Crc64Update = XzCrc64UpdateT4;
else
#endif
{
for (i = 256 * CRC64_NUM_TABLES - 1; i >= 256; i--)
{
UInt64 x = g_Crc64Table[(size_t)i - 256];
g_Crc64Table[i] = CRC_UINT64_SWAP(x);
}
g_Crc64Update = XzCrc64UpdateT1_BeT4;
}
}
#endif
}
+26
View File
@@ -0,0 +1,26 @@
/* XzCrc64.h -- CRC64 calculation
2013-01-18 : Igor Pavlov : Public domain */
#ifndef __XZ_CRC64_H
#define __XZ_CRC64_H
#include <stddef.h>
#include "7zTypes.h"
EXTERN_C_BEGIN
extern UInt64 g_Crc64Table[];
void MY_FAST_CALL Crc64GenerateTable(void);
#define CRC64_INIT_VAL UINT64_CONST(0xFFFFFFFFFFFFFFFF)
#define CRC64_GET_DIGEST(crc) ((crc) ^ CRC64_INIT_VAL)
#define CRC64_UPDATE_BYTE(crc, b) (g_Crc64Table[((crc) ^ (b)) & 0xFF] ^ ((crc) >> 8))
UInt64 MY_FAST_CALL Crc64Update(UInt64 crc, const void *data, size_t size);
UInt64 MY_FAST_CALL Crc64Calc(const void *data, size_t size);
EXTERN_C_END
#endif
+69
View File
@@ -0,0 +1,69 @@
/* XzCrc64Opt.c -- CRC64 calculation
2017-06-30 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include "CpuArch.h"
#ifndef MY_CPU_BE
#define CRC64_UPDATE_BYTE_2(crc, b) (table[((crc) ^ (b)) & 0xFF] ^ ((crc) >> 8))
UInt64 MY_FAST_CALL XzCrc64UpdateT4(UInt64 v, const void *data, size_t size, const UInt64 *table)
{
const Byte *p = (const Byte *)data;
for (; size > 0 && ((unsigned)(ptrdiff_t)p & 3) != 0; size--, p++)
v = CRC64_UPDATE_BYTE_2(v, *p);
for (; size >= 4; size -= 4, p += 4)
{
UInt32 d = (UInt32)v ^ *(const UInt32 *)p;
v = (v >> 32)
^ (table + 0x300)[((d ) & 0xFF)]
^ (table + 0x200)[((d >> 8) & 0xFF)]
^ (table + 0x100)[((d >> 16) & 0xFF)]
^ (table + 0x000)[((d >> 24))];
}
for (; size > 0; size--, p++)
v = CRC64_UPDATE_BYTE_2(v, *p);
return v;
}
#endif
#ifndef MY_CPU_LE
#define CRC_UINT64_SWAP(v) \
((v >> 56) \
| ((v >> 40) & ((UInt64)0xFF << 8)) \
| ((v >> 24) & ((UInt64)0xFF << 16)) \
| ((v >> 8) & ((UInt64)0xFF << 24)) \
| ((v << 8) & ((UInt64)0xFF << 32)) \
| ((v << 24) & ((UInt64)0xFF << 40)) \
| ((v << 40) & ((UInt64)0xFF << 48)) \
| ((v << 56)))
#define CRC64_UPDATE_BYTE_2_BE(crc, b) (table[(Byte)((crc) >> 56) ^ (b)] ^ ((crc) << 8))
UInt64 MY_FAST_CALL XzCrc64UpdateT1_BeT4(UInt64 v, const void *data, size_t size, const UInt64 *table)
{
const Byte *p = (const Byte *)data;
table += 0x100;
v = CRC_UINT64_SWAP(v);
for (; size > 0 && ((unsigned)(ptrdiff_t)p & 3) != 0; size--, p++)
v = CRC64_UPDATE_BYTE_2_BE(v, *p);
for (; size >= 4; size -= 4, p += 4)
{
UInt32 d = (UInt32)(v >> 32) ^ *(const UInt32 *)p;
v = (v << 32)
^ (table + 0x000)[((d ) & 0xFF)]
^ (table + 0x100)[((d >> 8) & 0xFF)]
^ (table + 0x200)[((d >> 16) & 0xFF)]
^ (table + 0x300)[((d >> 24))];
}
for (; size > 0; size--, p++)
v = CRC64_UPDATE_BYTE_2_BE(v, *p);
return CRC_UINT64_SWAP(v);
}
#endif
+2766
View File
File diff suppressed because it is too large Load Diff
+1329
View File
File diff suppressed because it is too large Load Diff
+60
View File
@@ -0,0 +1,60 @@
/* XzEnc.h -- Xz Encode
2017-06-27 : Igor Pavlov : Public domain */
#ifndef __XZ_ENC_H
#define __XZ_ENC_H
#include "Lzma2Enc.h"
#include "Xz.h"
EXTERN_C_BEGIN
#define XZ_PROPS__BLOCK_SIZE__AUTO LZMA2_ENC_PROPS__BLOCK_SIZE__AUTO
#define XZ_PROPS__BLOCK_SIZE__SOLID LZMA2_ENC_PROPS__BLOCK_SIZE__SOLID
typedef struct
{
UInt32 id;
UInt32 delta;
UInt32 ip;
int ipDefined;
} CXzFilterProps;
void XzFilterProps_Init(CXzFilterProps *p);
typedef struct
{
CLzma2EncProps lzma2Props;
CXzFilterProps filterProps;
unsigned checkId;
UInt64 blockSize;
int numBlockThreads_Reduced;
int numBlockThreads_Max;
int numTotalThreads;
int forceWriteSizesInHeader;
UInt64 reduceSize;
} CXzProps;
void XzProps_Init(CXzProps *p);
typedef void * CXzEncHandle;
CXzEncHandle XzEnc_Create(ISzAllocPtr alloc, ISzAllocPtr allocBig);
void XzEnc_Destroy(CXzEncHandle p);
SRes XzEnc_SetProps(CXzEncHandle p, const CXzProps *props);
void XzEnc_SetDataSize(CXzEncHandle p, UInt64 expectedDataSiize);
SRes XzEnc_Encode(CXzEncHandle p, ISeqOutStream *outStream, ISeqInStream *inStream, ICompressProgress *progress);
SRes Xz_Encode(ISeqOutStream *outStream, ISeqInStream *inStream,
const CXzProps *props, ICompressProgress *progress);
SRes Xz_EncodeEmpty(ISeqOutStream *outStream);
EXTERN_C_END
#endif
+319
View File
@@ -0,0 +1,319 @@
/* XzIn.c - Xz input
2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
#include <string.h>
#include "7zCrc.h"
#include "CpuArch.h"
#include "Xz.h"
/*
#define XZ_FOOTER_SIG_CHECK(p) (memcmp((p), XZ_FOOTER_SIG, XZ_FOOTER_SIG_SIZE) == 0)
*/
#define XZ_FOOTER_SIG_CHECK(p) ((p)[0] == XZ_FOOTER_SIG_0 && (p)[1] == XZ_FOOTER_SIG_1)
SRes Xz_ReadHeader(CXzStreamFlags *p, ISeqInStream *inStream)
{
Byte sig[XZ_STREAM_HEADER_SIZE];
RINOK(SeqInStream_Read2(inStream, sig, XZ_STREAM_HEADER_SIZE, SZ_ERROR_NO_ARCHIVE));
if (memcmp(sig, XZ_SIG, XZ_SIG_SIZE) != 0)
return SZ_ERROR_NO_ARCHIVE;
return Xz_ParseHeader(p, sig);
}
#define READ_VARINT_AND_CHECK(buf, pos, size, res) \
{ unsigned s = Xz_ReadVarInt(buf + pos, size - pos, res); \
if (s == 0) return SZ_ERROR_ARCHIVE; pos += s; }
SRes XzBlock_ReadHeader(CXzBlock *p, ISeqInStream *inStream, BoolInt *isIndex, UInt32 *headerSizeRes)
{
Byte header[XZ_BLOCK_HEADER_SIZE_MAX];
unsigned headerSize;
*headerSizeRes = 0;
RINOK(SeqInStream_ReadByte(inStream, &header[0]));
headerSize = (unsigned)header[0];
if (headerSize == 0)
{
*headerSizeRes = 1;
*isIndex = True;
return SZ_OK;
}
*isIndex = False;
headerSize = (headerSize << 2) + 4;
*headerSizeRes = headerSize;
RINOK(SeqInStream_Read(inStream, header + 1, headerSize - 1));
return XzBlock_Parse(p, header);
}
#define ADD_SIZE_CHECK(size, val) \
{ UInt64 newSize = size + (val); if (newSize < size) return XZ_SIZE_OVERFLOW; size = newSize; }
UInt64 Xz_GetUnpackSize(const CXzStream *p)
{
UInt64 size = 0;
size_t i;
for (i = 0; i < p->numBlocks; i++)
ADD_SIZE_CHECK(size, p->blocks[i].unpackSize);
return size;
}
UInt64 Xz_GetPackSize(const CXzStream *p)
{
UInt64 size = 0;
size_t i;
for (i = 0; i < p->numBlocks; i++)
ADD_SIZE_CHECK(size, (p->blocks[i].totalSize + 3) & ~(UInt64)3);
return size;
}
/*
SRes XzBlock_ReadFooter(CXzBlock *p, CXzStreamFlags f, ISeqInStream *inStream)
{
return SeqInStream_Read(inStream, p->check, XzFlags_GetCheckSize(f));
}
*/
static SRes Xz_ReadIndex2(CXzStream *p, const Byte *buf, size_t size, ISzAllocPtr alloc)
{
size_t numBlocks, pos = 1;
UInt32 crc;
if (size < 5 || buf[0] != 0)
return SZ_ERROR_ARCHIVE;
size -= 4;
crc = CrcCalc(buf, size);
if (crc != GetUi32(buf + size))
return SZ_ERROR_ARCHIVE;
{
UInt64 numBlocks64;
READ_VARINT_AND_CHECK(buf, pos, size, &numBlocks64);
numBlocks = (size_t)numBlocks64;
if (numBlocks != numBlocks64 || numBlocks * 2 > size)
return SZ_ERROR_ARCHIVE;
}
Xz_Free(p, alloc);
if (numBlocks != 0)
{
size_t i;
p->numBlocks = numBlocks;
p->blocks = (CXzBlockSizes *)ISzAlloc_Alloc(alloc, sizeof(CXzBlockSizes) * numBlocks);
if (!p->blocks)
return SZ_ERROR_MEM;
for (i = 0; i < numBlocks; i++)
{
CXzBlockSizes *block = &p->blocks[i];
READ_VARINT_AND_CHECK(buf, pos, size, &block->totalSize);
READ_VARINT_AND_CHECK(buf, pos, size, &block->unpackSize);
if (block->totalSize == 0)
return SZ_ERROR_ARCHIVE;
}
}
while ((pos & 3) != 0)
if (buf[pos++] != 0)
return SZ_ERROR_ARCHIVE;
return (pos == size) ? SZ_OK : SZ_ERROR_ARCHIVE;
}
static SRes Xz_ReadIndex(CXzStream *p, ILookInStream *stream, UInt64 indexSize, ISzAllocPtr alloc)
{
SRes res;
size_t size;
Byte *buf;
if (indexSize > ((UInt32)1 << 31))
return SZ_ERROR_UNSUPPORTED;
size = (size_t)indexSize;
if (size != indexSize)
return SZ_ERROR_UNSUPPORTED;
buf = (Byte *)ISzAlloc_Alloc(alloc, size);
if (!buf)
return SZ_ERROR_MEM;
res = LookInStream_Read2(stream, buf, size, SZ_ERROR_UNSUPPORTED);
if (res == SZ_OK)
res = Xz_ReadIndex2(p, buf, size, alloc);
ISzAlloc_Free(alloc, buf);
return res;
}
static SRes LookInStream_SeekRead_ForArc(ILookInStream *stream, UInt64 offset, void *buf, size_t size)
{
RINOK(LookInStream_SeekTo(stream, offset));
return LookInStream_Read(stream, buf, size);
/* return LookInStream_Read2(stream, buf, size, SZ_ERROR_NO_ARCHIVE); */
}
static SRes Xz_ReadBackward(CXzStream *p, ILookInStream *stream, Int64 *startOffset, ISzAllocPtr alloc)
{
UInt64 indexSize;
Byte buf[XZ_STREAM_FOOTER_SIZE];
UInt64 pos = *startOffset;
if ((pos & 3) != 0 || pos < XZ_STREAM_FOOTER_SIZE)
return SZ_ERROR_NO_ARCHIVE;
pos -= XZ_STREAM_FOOTER_SIZE;
RINOK(LookInStream_SeekRead_ForArc(stream, pos, buf, XZ_STREAM_FOOTER_SIZE));
if (!XZ_FOOTER_SIG_CHECK(buf + 10))
{
UInt32 total = 0;
pos += XZ_STREAM_FOOTER_SIZE;
for (;;)
{
size_t i;
#define TEMP_BUF_SIZE (1 << 10)
Byte temp[TEMP_BUF_SIZE];
i = (pos > TEMP_BUF_SIZE) ? TEMP_BUF_SIZE : (size_t)pos;
pos -= i;
RINOK(LookInStream_SeekRead_ForArc(stream, pos, temp, i));
total += (UInt32)i;
for (; i != 0; i--)
if (temp[i - 1] != 0)
break;
if (i != 0)
{
if ((i & 3) != 0)
return SZ_ERROR_NO_ARCHIVE;
pos += i;
break;
}
if (pos < XZ_STREAM_FOOTER_SIZE || total > (1 << 16))
return SZ_ERROR_NO_ARCHIVE;
}
if (pos < XZ_STREAM_FOOTER_SIZE)
return SZ_ERROR_NO_ARCHIVE;
pos -= XZ_STREAM_FOOTER_SIZE;
RINOK(LookInStream_SeekRead_ForArc(stream, pos, buf, XZ_STREAM_FOOTER_SIZE));
if (!XZ_FOOTER_SIG_CHECK(buf + 10))
return SZ_ERROR_NO_ARCHIVE;
}
p->flags = (CXzStreamFlags)GetBe16(buf + 8);
if (!XzFlags_IsSupported(p->flags))
return SZ_ERROR_UNSUPPORTED;
if (GetUi32(buf) != CrcCalc(buf + 4, 6))
return SZ_ERROR_ARCHIVE;
indexSize = ((UInt64)GetUi32(buf + 4) + 1) << 2;
if (pos < indexSize)
return SZ_ERROR_ARCHIVE;
pos -= indexSize;
RINOK(LookInStream_SeekTo(stream, pos));
RINOK(Xz_ReadIndex(p, stream, indexSize, alloc));
{
UInt64 totalSize = Xz_GetPackSize(p);
if (totalSize == XZ_SIZE_OVERFLOW
|| totalSize >= ((UInt64)1 << 63)
|| pos < totalSize + XZ_STREAM_HEADER_SIZE)
return SZ_ERROR_ARCHIVE;
pos -= (totalSize + XZ_STREAM_HEADER_SIZE);
RINOK(LookInStream_SeekTo(stream, pos));
*startOffset = pos;
}
{
CXzStreamFlags headerFlags;
CSecToRead secToRead;
SecToRead_CreateVTable(&secToRead);
secToRead.realStream = stream;
RINOK(Xz_ReadHeader(&headerFlags, &secToRead.vt));
return (p->flags == headerFlags) ? SZ_OK : SZ_ERROR_ARCHIVE;
}
}
/* ---------- Xz Streams ---------- */
void Xzs_Construct(CXzs *p)
{
p->num = p->numAllocated = 0;
p->streams = 0;
}
void Xzs_Free(CXzs *p, ISzAllocPtr alloc)
{
size_t i;
for (i = 0; i < p->num; i++)
Xz_Free(&p->streams[i], alloc);
ISzAlloc_Free(alloc, p->streams);
p->num = p->numAllocated = 0;
p->streams = 0;
}
UInt64 Xzs_GetNumBlocks(const CXzs *p)
{
UInt64 num = 0;
size_t i;
for (i = 0; i < p->num; i++)
num += p->streams[i].numBlocks;
return num;
}
UInt64 Xzs_GetUnpackSize(const CXzs *p)
{
UInt64 size = 0;
size_t i;
for (i = 0; i < p->num; i++)
ADD_SIZE_CHECK(size, Xz_GetUnpackSize(&p->streams[i]));
return size;
}
/*
UInt64 Xzs_GetPackSize(const CXzs *p)
{
UInt64 size = 0;
size_t i;
for (i = 0; i < p->num; i++)
ADD_SIZE_CHECK(size, Xz_GetTotalSize(&p->streams[i]));
return size;
}
*/
SRes Xzs_ReadBackward(CXzs *p, ILookInStream *stream, Int64 *startOffset, ICompressProgress *progress, ISzAllocPtr alloc)
{
Int64 endOffset = 0;
RINOK(ILookInStream_Seek(stream, &endOffset, SZ_SEEK_END));
*startOffset = endOffset;
for (;;)
{
CXzStream st;
SRes res;
Xz_Construct(&st);
res = Xz_ReadBackward(&st, stream, startOffset, alloc);
st.startOffset = *startOffset;
RINOK(res);
if (p->num == p->numAllocated)
{
size_t newNum = p->num + p->num / 4 + 1;
Byte *data = (Byte *)ISzAlloc_Alloc(alloc, newNum * sizeof(CXzStream));
if (!data)
return SZ_ERROR_MEM;
p->numAllocated = newNum;
if (p->num != 0)
memcpy(data, p->streams, p->num * sizeof(CXzStream));
ISzAlloc_Free(alloc, p->streams);
p->streams = (CXzStream *)data;
}
p->streams[p->num++] = st;
if (*startOffset == 0)
break;
RINOK(LookInStream_SeekTo(stream, *startOffset));
if (progress && ICompressProgress_Progress(progress, endOffset - *startOffset, (UInt64)(Int64)-1) != SZ_OK)
return SZ_ERROR_PROGRESS;
}
return SZ_OK;
}
+241
View File
@@ -0,0 +1,241 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<ItemGroup>
<ClInclude Include="7z\7z.h" />
<ClInclude Include="7z\7zAlloc.h" />
<ClInclude Include="7z\7zBuf.h" />
<ClInclude Include="7z\7zCrc.h" />
<ClInclude Include="7z\7zFile.h" />
<ClInclude Include="7z\7zTypes.h" />
<ClInclude Include="7z\7zVersion.h" />
<ClInclude Include="7z\Aes.h" />
<ClInclude Include="7z\Alloc.h" />
<ClInclude Include="7z\Bcj2.h" />
<ClInclude Include="7z\Bra.h" />
<ClInclude Include="7z\Compiler.h" />
<ClInclude Include="7z\CpuArch.h" />
<ClInclude Include="7z\Delta.h" />
<ClInclude Include="7z\DllSecur.h" />
<ClInclude Include="7z\LzFind.h" />
<ClInclude Include="7z\LzFindMt.h" />
<ClInclude Include="7z\LzHash.h" />
<ClInclude Include="7z\Lzma2Dec.h" />
<ClInclude Include="7z\Lzma2DecMt.h" />
<ClInclude Include="7z\Lzma2Enc.h" />
<ClInclude Include="7z\Lzma86.h" />
<ClInclude Include="7z\LzmaDec.h" />
<ClInclude Include="7z\LzmaEnc.h" />
<ClInclude Include="7z\LzmaLib.h" />
<ClInclude Include="7z\MtCoder.h" />
<ClInclude Include="7z\MtDec.h" />
<ClInclude Include="7z\Ppmd.h" />
<ClInclude Include="7z\Ppmd7.h" />
<ClInclude Include="7z\Precomp.h" />
<ClInclude Include="7z\RotateDefs.h" />
<ClInclude Include="7z\Sha256.h" />
<ClInclude Include="7z\Sort.h" />
<ClInclude Include="7z\Threads.h" />
<ClInclude Include="7z\Xz.h" />
<ClInclude Include="7z\XzCrc64.h" />
<ClInclude Include="7z\XzEnc.h" />
</ItemGroup>
<ItemGroup>
<ClCompile Include="7z\7zAlloc.c" />
<ClCompile Include="7z\7zArcIn.c" />
<ClCompile Include="7z\7zBuf.c" />
<ClCompile Include="7z\7zBuf2.c" />
<ClCompile Include="7z\7zCrc.c" />
<ClCompile Include="7z\7zCrcOpt.c" />
<ClCompile Include="7z\7zDec.c" />
<ClCompile Include="7z\7zFile.c" />
<ClCompile Include="7z\7zStream.c" />
<ClCompile Include="7z\Aes.c" />
<ClCompile Include="7z\AesOpt.c" />
<ClCompile Include="7z\Alloc.c" />
<ClCompile Include="7z\Bcj2.c" />
<ClCompile Include="7z\Bcj2Enc.c" />
<ClCompile Include="7z\Bra.c" />
<ClCompile Include="7z\Bra86.c" />
<ClCompile Include="7z\BraIA64.c" />
<ClCompile Include="7z\CpuArch.c" />
<ClCompile Include="7z\Delta.c" />
<ClCompile Include="7z\DllSecur.c" />
<ClCompile Include="7z\LzFind.c" />
<ClCompile Include="7z\LzFindMt.c" />
<ClCompile Include="7z\Lzma2Dec.c" />
<ClCompile Include="7z\Lzma2DecMt.c" />
<ClCompile Include="7z\Lzma2Enc.c" />
<ClCompile Include="7z\Lzma86Dec.c" />
<ClCompile Include="7z\Lzma86Enc.c" />
<ClCompile Include="7z\LzmaDec.c" />
<ClCompile Include="7z\LzmaEnc.c" />
<ClCompile Include="7z\LzmaLib.c" />
<ClCompile Include="7z\MtCoder.c" />
<ClCompile Include="7z\MtDec.c" />
<ClCompile Include="7z\Ppmd7.c" />
<ClCompile Include="7z\Ppmd7Dec.c" />
<ClCompile Include="7z\Ppmd7Enc.c" />
<ClCompile Include="7z\Sha256.c" />
<ClCompile Include="7z\Sort.c" />
<ClCompile Include="7z\Threads.c" />
<ClCompile Include="7z\Xz.c" />
<ClCompile Include="7z\XzCrc64.c" />
<ClCompile Include="7z\XzCrc64Opt.c" />
<ClCompile Include="7z\XzDec.c" />
<ClCompile Include="7z\XzEnc.c" />
<ClCompile Include="7z\XzIn.c" />
</ItemGroup>
<PropertyGroup Label="Globals">
<VCProjectVersion>16.0</VCProjectVersion>
<ProjectGuid>{5B146DEA-9ACE-4D32-A7FD-3F42464DD69C}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>My7zlib</RootNamespace>
</PropertyGroup>
<Import Project="..\common_default.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<Import Project="..\common_default_macros.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="Shared">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup>
<OutDir>$(SolutionDir)lib/$(Configuration)-$(Platform)/</OutDir>
<IntDir>$(SolutionDir)tmp\$(ProjectName)-$(Configuration)-$(Platform)/</IntDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<LinkIncremental>false</LinkIncremental>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<LinkIncremental>true</LinkIncremental>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ClCompile>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<SDLCheck>true</SDLCheck>
<PreprocessorDefinitions>NDEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<ConformanceMode>true</ConformanceMode>
<PrecompiledHeaderFile>pch.h</PrecompiledHeaderFile>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>Use</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>Disabled</Optimization>
<SDLCheck>true</SDLCheck>
<PreprocessorDefinitions>WIN32;_DEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<ConformanceMode>true</ConformanceMode>
<PrecompiledHeaderFile>pch.h</PrecompiledHeaderFile>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<Optimization>Disabled</Optimization>
<SDLCheck>true</SDLCheck>
<PreprocessorDefinitions>_DEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<ConformanceMode>true</ConformanceMode>
<PrecompiledHeaderFile>pch.h</PrecompiledHeaderFile>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<PrecompiledHeader>Use</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<SDLCheck>true</SDLCheck>
<PreprocessorDefinitions>WIN32;NDEBUG;_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<ConformanceMode>true</ConformanceMode>
<PrecompiledHeaderFile>pch.h</PrecompiledHeaderFile>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<GenerateDebugInformation>true</GenerateDebugInformation>
</Link>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>

Some files were not shown because too many files have changed in this diff Show More